Software Engineering Challenges and Best Practices for Multi-Institutional Scientific Software Development
From AcaWiki
Citation: Keith Beattie, Dan Gunter (2021/08) Software Engineering Challenges and Best Practices for Multi-Institutional Scientific Software Development. IDEAS Productivity Project Webinar (RSS)
Internet Archive Scholar (search for fulltext): Software Engineering Challenges and Best Practices for Multi-Institutional Scientific Software Development
Download: http://ideas-productivity.org/wordpress/wp-content/uploads/2021/08/hpcbp055-sebp4ssd.pdf
Tagged: Computer Science
(RSS) computational science (RSS)
Axes of difficulty
- Team size
- Geographic distribution
- SW dev experience/exposure
- Audience size
- Contributor time allocation
- Collaboration scale
Scrum for computational science
- https://www.exascaleproject.org/event/agile/
- Not all of Scrum/Agile/Kanban will apply directly to academia (multiple timezones, multiple projects)
Practices
- Scheduled meetings
- Daily may not be possible
- Date-driven rather than feature-driven releases
- Iterative incremental improvements (to the project and the process)
- Internal education
- Set up dev env (build, test, gen docs)
- External education
- Set up user env
- Diataxis: https://diataxis.fr/
- Version control, issues, PRs, code review
- Don't make people feel punished or put on a hotseat.
- Automated tests (lints, style guides, static analysis, performance testing, fuzzing, code coverage, documentation coverage, documentation tests)
- Save seed for random inputs
- But also run locally
- Continuous deployment: Can set up nightly/weekly tests that are too expensive to test every time
Pearls of Wisdom
- Cooking for yourself : solo engineering :: cooking for a dinner party : team engineering :: opening a restaurant : engineering a public package. They are all cooking, but they are very different in planning and preperation.
- Not all tech problems have tech solutions
- Not all social problems have social solutions
- Culture eats process for breakfast
- Often, you can't make people do things, so instead you have to convince them to.