I think the key to understanding research code for industrial software engineers is to accept that you are typically not building a product. You do not have customers as such. You are building software to prove a point.

As such, the majority of code that you write as a researcher is more akin to the throw-away prototypes and mockups that you (in industry) often write in the early phases of a project. As you are certainly aware, even in industry these mockups have quite different properties than the final software. They primarily need to:

• Have exactly the features that you want to show to the customer. Not more, not less. Typically, all the boring standard features for the domain are omitted.
• Need to get done quickly. Both you and the customer know that the prototype will be thrown away anyway, so it does not matter whether the code is maintainable.
• Need to be easy to extend and adapt, optimally live during the demo.

Essentially the same properties are also useful for most throwaway research code. You do not want to build features that you do not need. You do not want to waste time writing e.g., maintainable code if you know that it will not be maintained. You want to use an environment that reduces the amount of boilerplate code and setup, and which maybe auto-generates a lot of code for you that is "good enough" for your demonstrator (Ruby on Rails and its scaffolding features come to mind).

My career progression is dependent on me writing "bad" code!?

No, it depends on you writing code fit for purpose. Just like in industry. In industry and academia nobody applauds you for software qualities that are not needed. Try to reconsider what the point of the code is that you are writing. If you plan to release your code as open source software and you expect it to be picked up by other people across the world, then go nuts - use all the engineering techniques you have also used in industry to build the best product you can. If your goal is to evaluate this one algorithm or principle for your conference, and then throw away the code, then you can also live happily without writing a single unit test without feeling like a fraud at all.

EDIT: of course this does not mean that it is acceptable to write code where you are unsure whether you have implemented said algorithm correctly. Ensuring that what you have indeed shown what you claim to have shown is mandatory, especially in research code.

I am a researcher and self-taught developer. I have done substantial projects which were primarily software based. Although my work is far from the most "hardcore" stuff that's out there in terms of complexity and scale, the projects were big enough that naive mistakes (eg. not using version control or poorly documenting code) were very painful. I ended up learning quite a few "best practices" through trial and error.

I have also been on the receiving end of "unmaintanable code passed down to fellow researcher":

my industrial experience has been a hindrance in my research

No, you have basically come from a civilized environment that solved these problems decades ago into one that is stuck in the stone age in terms of software development hygiene. Scientists still code like it's the 60s. Of course you feel a conflict, but the fault is not with you.

In industry, code needs to be (ideally): maintainable, bug-free, refactored, well-documented, rigorously tested

Let's say the speaker at a scientific conference, while describing the computational part of his research, said one of the following:

"The code I wrote for this research is, admittedly..."

• ...unmaintainable (and good luck building on my research!)
• ...full of bugs (and I have no idea if the output is even correct!)
• ...unreadable spagetti (and I don't even know how it works, let alone if it does so correctly!)
• ...undocumented (and all the mistakes are obfuscated from reviewers!)
• ...not tested (so god knows if it does what I say/think it does!)

Do you expect the audience to react with anything but scorn and outrage? If I heard such a thing, I would not believe anything this person published ever again.

In academia, the goal is (...) in the shortest possible time.

Yes, but "no shorter". You don't skip vital control experiments because "controls take time". You can't skimp on code quality for very similar reasons.

There seems to be no motivation to write [good code]

Because this is an endemic problem of academia. Although computers have been used in science for decades, it seems that algorithms have only become an important part of research in the last decade or so (perhaps because of "big data"). When you base your research on code, that code must be good quality. It is not enough to simply crank out some buggy write-only script and call it a day. The software development community has figured all of this out long ago, but academia has not yet caught on - I think the reason is that most scientists do not have a formal background in software development, and there have not been enough huge scandals in research caused by bad programming practice (eg. key results of a high-profile paper turn out to be artifacts caused by bugs).

Consider how, in many disciplines, reviewers will not even ask about the source code of your computation-heavy paper. How can they evaluate, then, the validity of your results? They cannot, and this is a failure of the peer review model as it currently exists.

Sorry to go on a rant, but basically, it's like this: As you know, there are very good reasons for writing quality code, even if no one is watching over your shoulder. In science, currently it so happens that nobody cares if your code is good or not. But this should not be a reason for you to not write good code anyway - the reasons for writing good code in the industry still largely apply to science.

Unfortunately, you may not be rewarded for your extra work. You may even be punished, because as you say, good code takes longer, and others may not see beyond that. Your PI or colleagues may not understand why you are so much slower. The best you can do is explain to them the need for good practices.

Obviously, there are exceptions. For instance, you may not need to worry about portability or backwards compatibility with old versions of the OS for code that is meant to run on a dedicated lab computer (although it is undesirable to write your code such that it only runs in a very exotic environment that other scientists will not be able to easily reconstruct). But by and large, I find that industry practices still apply, and the exceptions can be easily detected by applying a modicum of critical thought. That said, there is also a helpful publication called "Best Practices for Scientific Computing" which examines this matter in detail.

Ultimately, it is an ethical decision you must make. Do you care about doing good science above all else? Follow best practices. Do you want to cut corners that you shouldn't (in an ethical sense), to save time or avoid friction with co-workers? I couldn't recommend you to do this, on principle. But obviously many people do, and perhaps in practice, some scientists are forced to do it - although then again, does being unable to do good science by circumstance excuse bad science?

Also, like I said, I think part of the problem is that there haven't been any big scandals. If you do skimp on code quality, there's a chance it will catch up with you. You might even end up being one those big scandals. Admittedly, the risk is probably small... But, I think you can see my point.

I either spend too long making (unnecessarily) getting my "research" code to industry-quality

Don't. Make it as good as possible within the suggested timeframe. Aiming for 100% perfection that will require double the time is not worth it. In this sense, research is exactly like industry.

Consequently, the "academic" code I've written is poor quality

That does not mean all academic code is of low quality. If you check papers on algorithms conferences or parallel processing systems, you will see that the developers have thought even excruciating details, like reordering of data for fewer cache misses, SIMD, GPU programming, SSD storage etc. Usually advanced CS algorithms research is some years ahead in adopting new methods, hardware techniques before any of those techniques actually hit the industry. On the other hand, in more theoretical CS conferences code is mainly a tool and as such, it does not have to be cutting edge. So, the quality is related to the audience of your product code (exactly like industry).

Does anyone know of formal methodologies for "research" code?

I have never heard on any methods especially tailored for research code. Still, you can use the practices from your industry background (when they actually accelerate your process of writing the software). For example a versioning system accelerates development and minimizes errors / losses of data. On the other hand UNIT tests take a lot of time, which you might not have. A informal wiki for bugs, documentation, features might worth the extra time, since it also accelerates writing the actual research paper. Contrarily, a full blown bug database (bugzilla) might not worth the extra time and effort.

So, stick to those industry methods, techniques you know will save you time on the long run and will improve your software but without taking all of your time. Finding a middle ground is always the best solution.

The tension between writing research code for industry and academia can be challenging. In research, the primary focus is on advancing knowledge and producing publishable results, which can lead to a mismatch with industry best practices. Here are some strategies to help you balance the two: 1. **Separate development and research goals**: Consider using separate repositories or codebases for your research and development projects. This way, you can maintain industry-quality code for your professional projects and prioritize research goals for your academic work. 2. **Use lightweight documentation and testing**: While comprehensive documentation and rigorous testing are essential in industry, you can adapt to research contexts by using lightweight tools and methods. For example, you can use Markdown or simple README files for documentation, and basic unit testing or integration testing to ensure the code runs correctly. 3. **Focus on reproducibility**: In research, reproducibility is crucial. Instead of striving for industry-quality code, focus on making your code reproducible, transparent, and easily maintainable. This can include using open-source tools, providing detailed explanations of your methods, and sharing your code and data with the research community. 4. **Collaborate with industry peers**: Join online communities, forums, or research groups focused on software engineering and research. Share your experiences, ask for advice, and learn from peers who may be facing similar challenges. 5. **Develop a "good enough" mindset**: In research, it's often necessary to prioritize speed and agility over perfection. Acknowledge that your research code may not meet industry standards, but still strive to make it functional, readable, and maintainable enough to support your research goals. 6. **Consider using research-specific tools and frameworks**: Explore research-focused tools, such as Jupyter Notebooks, PyTorch, or TensorFlow, which can help streamline your workflow and reduce the need for extensive documentation and testing. By adopting these strategies, you can navigate the tension between research and industry goals, producing code that meets your research needs while still maintaining a level of quality and maintainability.