Having spending a decade with different software projects and development methodologies, I would conclude that they are somehow not really optimal in many real life Scenarios, independently if we speak about classical waterfall style software development, or about a rather modern agile or scrum method. In the following I just try to summarise a couple of points that I see important based on my experience.
1. Specification: Most software project methodology suppose that there is some kind of a specification for the software; it is relative good specified at waterfall however much more flexible at agile. However in life the first question of the customer if you already have something ready ? The second question is if you have at least something half-ready for prototyping. And the reason for that is that even creating a relative good specification even if it is agile takes time and money. Certainly it can be supported by the business analyst role, so it takes more money and time. As a consequence, custom software development is ususally the last choice for any corporate purchasing process. Customers usually prefer to get something ready or at least-half ready and making modification on that.
2. Architecture: There are a lot of general frameworks that delivers a lot of use-cases out of the box, other frameworks provide self service use-case development even by people without prior development knowledge. In this sense one of the most important step for every software development project is to choose the right software architecture.
3. Agility: Agility can be interpreted only within a certain software architecture. Each architecture can provide several use-cases out of the box or easy to set up, however others are very difficult to implement. One of the biggest problem of each project if the basic architecture should be redesigned during the project.
4. Lean software development: Lean software development can be interpreted as a way for delivering usable software for the end customer as fast and as efficient as possible. In this sense it makes sense to evaluate if part of the specification or respecification at a new release and the related development can be supported by machine learning algorithms. Certainly the architecture hat to support such an improvement as well, however it is not difficult to imagine that for instance the user interface structure is automaticaly adapted just by analysing the usage of the software itself. There are initiatives for that for exmape The Grid.
5. Resue - Resell: Most software companies do not want to sell a project only once but several times, even if it is not meant to be develpped as a product for the first run. In this sense one of the most important question for each software project is reuse and resell: How can be the software project sold for several customers, how can be an implemented business-case used in several projects or even on several platforms.
