How can Agile methodologies, CI/CD be applied to massive, monolithic software projects with 100s/1000s of developers?

HOW TO -️ October 18, 2021

Using continuous integration represents an (almost) mandatory requirement for a software project to be agile. And without CI there can be no continuous delivery/deployment.

But CI can face scalability challenges, see How can continuous integration scale for very large projects/teams?.

Splitting work on multiple smaller pre-integration branches would allow work on those branches to be agile, but then merging of those pre-integration branches needs to be handled. And those would be massive, unpredictable merges, rather difficult to plan and incorporate into an Agile framework. Until the merge is actually performed one can't really tell if it'll take 2 hours, 2 days or 2 entire sprints to complete.

So how can Agile methodologies still be applied to massive, monolithic software projects with 100s/1000s of developers?

Answer

This question is mixing two big topics, one is Agile for big teams, another is Ci/CD for many/big projects. It would really benefit if you could split this into two questions at least and not muddle one with the other. "Branches" are not Agile, people and teams and work items is where Agile is scoped. But these topics are, IMHO, very much related. Looking them together, as a system (which is the reality, if you think about it) can offer opportunities for optimisations which can't be seen if looking at each of them in isolation. What takes 2 hours/days, deployment, automated testing or manual merging? @kenorb: the whole thing - I mean you can't really declare the branch merge successful until the SCM merge is done and committed, artifacts are successfully built and tested and deployment actually proves that the whole thing is acceptable (or Done?)

Initializing...