Almost one year ago machine learning (ML) and software engineering (SE) experts came together in Montréal to discuss challenges, new insights, and practical ideas regarding the engineering of ML and AI-based systems. This was the first instalment of the Software Engineering for Machine Learning Applications (SEMLA) symposium. Key challenges discussed included when ML/AI should be used, the accuracy of systems built using ML and AI models, the testing of those systems, industrial applications of AI, and the rift between the ML and SE communities. Sadly, recently we learned that expressed concern were real. Regulations, applied processes and practices were not sufficient to avoid the Boeing 737 MAX crashes.
The 737 MAX is just the last of a series of cases were assumptions were flawed, verification and validation poorly done and oversight was missing or biased. This talk does a step back to the origin of the non testable program concept and the intrinsic limits of our working memory. While AI is becoming more and more pervasive there is an urgent need to define new sound ways to ascertain critical software qualities such as reliability or safety keeping in mind that data play a much bigger role. This require a shift in the traditional V&V paradigm, processes and people training.