Saturday, September 17, 2016

The business value of code elegance in the digital age


Introduction


I was invited two weeks ago to a great event hosted by EPITA entitled “The Elegance of Algorithms”. The host of the debate was Cedric Villani and I really enjoyed this evening of thought-provoking speeches. I was delighted to be invited to join and participate, since the “elegance of programming & algorithms” has been a pet topic of mine for the last 30 years.

I got this invitation thanks to Akim Demaille who remembered that the title of the original web page of the CLAIRE programming language is “The Art of Elegant Programming”. I designed CLAIRE as an open source project in the 90s, together with François Laburthe and a support group including Akim and Stephane Hadinger, as a language that would be ideally suited to write operations research agorithms. CLAIRE is itself the result of 10 years of experience writing compilers for more complex languages such as SPOKE and LAURE. I decided that too much effort had been spent on sophisticated features that programmers did not use much, and that we should rather focus on what made the language versatile and elegant. Our new goal for CLAIRE was to write “executable pseudo-code”, with a language that could be used to teach algorithms, which I did with CLAIRE for a number of years at ENS and Jussieu. Our measure of elegance for CLAIRE was, for instance, the number of lines necessary to express – in an elegant form – the Hungarian matching algorithm. The challenge was to be as compact/efficient as possible – using a high level of abstraction, in the spirit of APL – while staying close to natural language (not the spirit of APL).

There were a number of reasons to focus on elegance 20 years ago. I was already aware that software systems are alive and that a good programming language should facilitate the constant grooming of the code. I had also learned the hard way that only what is simple and elegant survives when generations of programmers work onto a software system. However, what was only a vague intuition then is now blatantly obvious today in the digital age. Which is why I decided to jot down some of the key ideas that I developed in my speech at this EPITA event. Simply said, this post is about the monetary value of software elegance, from architecture to code.


The Business Value of Elegance


1. Software in the digital age is about flows, not assets



The key marker of software in the digital age is the continuous change of software systems; the consequence is that we need to love our code. This is very different from the “system engineering” culture of a few decades ago when we promoted a “black box” approach. If software maintenance required to edit and modify your code constantly, it should be nice to look at ! “Source code is back” and the aesthetics of programming becomes important. Elegance is what helps software systems to evolve into the hands of successive teams of programmers, when “ugly code” is encapsulated and “worked around”, producing hidden complexity and not-so-hidden cost explosion.

This world of constant adaptation is the world of agile software methods, based on iteration. However, iteration produces junk, accumulation of things that quickly becomes useless. It is a law of nature that is not specific to software. Iteration must come hand in hand with refactoring. As a metaphor, we need to tend the garden: clean up, sort and reorganize. Elegance becomes a philosophy and an architecture principle. Gardening means to grow simplicity and to let the “system’s potential” emerge. Digital gardening is similar to Zen gardening : to discover what is permanent in a changing flow.

This vision of systems as flows will intensify in the digital feature when systems are built through the collaboration of AI and human intelligence. Today already, more and more digital algorithms are grown from data through machine learning. This is why Henri Verdier told us that “data is the new code” – I refer the reader to the report on Big Data published by the National Academy of Technologies. What is developed with digital systems are no longer algorithms in the classical sense but processes that grow the necessary algorithms. However, these processes are meta-programmed with code that needs to be elegant for the very same reasons.


2. Digital software is the fruit of collaboration


The act of sharing is critical to building high quality software in a flow environment. This is why code reviews play such an important role and why pair-programming is at the heart of extreme programming. Digital software is the result of team efforts. Code reviews must be enjoyable to be sustained; elegance becomes a Darwinian virtue of code that survives these reviews. During this dinner, I went as far as saying that “elegance is the fuel of code reviews”, as an emotional, aesthetic and collaborative experience.

Digital systems leverage the “power of the crowd” through open source software. We now live in the world of software ecosystems that are built by communities. Software platforms are grown by weaving fragments that get reused through sharing. The elegance of the software fragments is the cement of the open source platforms. The more elegant the code, more eyeballs are reading and proofing it. A law of the digital age is that quality comes from the multiplication of reuse.

Elegance is a dynamic property revealed in a communication process, not a static or aesthetic judgment. This is not new, a few centuries ago, Boileau told us that “Whatever we conceive well we express clearly, and words flow with ease”. Elegance matters to make communication, hence collaboration, easier. An interesting parallel may be made with Cedric Villani’s book, “Birth of a Theorem: A Mathematical Adventure”: scientific creativity is the result of a network whose paths are the laborious efforts – mathematicians in front of their white pages or programmers in from of their screens – to progress along a task, while the nodes are the communication moments when collaboration occurs. Cedric’s book is a brilliant illustration of the importance of the collaboration network. It is, therefore, not a surprise if elegance has been considered a virtue in Mathematics for ages.


3. Today’s main challenge for software is the increasing word complexity




A key ambition of elegance is to reach simplicity. It means not only to search for simplicity of the first attempt or the first creative result, but also the constant quest for continuous simplification, in the scientific tradition of Occam’s razor. Gaston Bachelard told us that “simplicity is the result of a long process of simplification”. Simplicity received a lot of attention during our evening discussion, from a mathematical, musical or software point of view.

Simplicity is a north star of digital system because of its antifragile nature. This great concept from Nassim Taleb means to be reinforced (versus broken) by the constant flow of unexpected and adversarial events. For instance, biological live systems are antifragile whereas most mechanical systems are fragile. Simple code is antifragile – compared to sophisticated one – because it is much more prone to evolve at the same rate as the environment’s change. Digital systems require this behavior; successful platform are built through a constant flow of adaptation. There is much deeper systemic truth here, that you may start to grasp when reading “Simple Rules for a Complex World” by  Donald SullKathleen and M. Eisenhardt.

A cardinal virtue of simplicity in the digital age is the reduction of inertia. This is a core lean principle from Taiichi Ohno. There is much more here than the simple application of Newton dynamics (the lower the mass, the higher the acceleration). Since I was talking to mathematicians, I was able to refer to Pollaczek-Khinchine ‘s formula which I have quoted often in my blog. Without going into the complexity of Jackson networks, this formula expresses the virtue of simplicity (reduction of variation) to ensure a faster response time. This is one of the many case where the intuition of lean and the mathematical models coincide. This translates into a key principle for digital: to continuously clean up the technical debt, since the overall velocity is inversely proportional to the bulk of digital assets.

Conclusion


Throughout this post, I have used the word “elegance” to combine two qualities: simplicity – from a systemic perspective – and aesthetic value, measured though communication and collaboration. The three previous sections could be summarized as:

  • Elegance is necessary to break the “time barrier” – allow long-lasting software improvements
  • Elegance is necessary to break the “distance barrier” – enabling the collaboration of remote viewpoints to produce better software
  • Elegance is necessary to break the “complexity barrier” – in order to design antifragile systems.


This is, on purpose, a conceptual and slightly pedantic way to express why elegance matters. I concluded with a much more forceful anecdote taken from a previous visit to Googleplex in Mountainview, where I have enjoyed reading a few good pages of tips about test-driven development, while standing in the restrooms. This is a vivid example of what a “code loving culture” can be. The silent drama, which is happening right now, is that the gap between those companies that have understood that “software is eating the world” and those who do not is growing. This is where we have come full circle to the main topics of this blog. I will refer the reader to Octo’s great book : “The Web Giants” where almost everything that I mentioned in this post resonate with the best practices of the most advanced software companies.


Code elegance is a fascinating topic and there is much more to say. For instance, it is interesting to try to characterize what makes a code elegant. Cedric Villani started his introduction by saying that elegance in mathematics was the combination of simplicity (concision), efficiency (something that really works) and surprise. The aesthetic value of surprise is a direct consequence of our emotional species-meta-learning strategy to value what is different from we expect (cf. Michio Kaku). I will end this post with a few ideas borrowed from Francois Laburthe, with an interesting reference to SOLID:
  • Elegant design borrows heavily on abstraction and genericity. However, units (from functions or classes to module) have a clear and well defined purpose.
  • Elegant design is mindful of waste, in the lean tradition. Everything is used, and constant refactoring tracks useless replication of features.
  • Elegant design is geared toward self-dissemination because it “welcomes you” through a self-explanatory structure. This is greatly helped by reification / introspection when design elements are self-aware.
  • Elegant design is open by nature – it embraces the philosophy of “fragment weaving” mentioned early on through facilitating (hence the importance of API) and selflessness (the design principle that the code will always be used by something more important than itself).






 
Technorati Profile