UPDATED – TALK NOW CANCELLED (BY GOOGLE CAMPUS LONDON), VERY SORRY! 🙁
(This is a little off-topic, but what the hey.)
When I’m not trying to crack unbroken historical ciphers, my day job is as a software developer: and over the last few years, I’ve got more and more frustrated with the abysmally low productivity of the whole industry.
And so I’ve been putting a lot of thought into how to make things better. And by “better” I don’t mean a mere 10% better, but instead something closer to 10x better – for example, how to write a hundred lines of new code that does the work of a thousand lines of old code. But note that this really isn’t about Computer Scientists devising the ‘perfect’ programming language, it’s about looking at the different activities software developers do and seeing how we can turbo-charge them all, almost beyond recognition.
Back in 2015, I called this “The Ten Times Manifesto” to try to give it some shape and direction, and discussed it with a whole load of people.
Now in 2018, it’s time for me to take The Ten Times Manifesto out on the road, to open up lots of new conversations with lots of new people and see where the whole constellation of ideas leads. I genuinely think this is the time that software development has to make a step change, to turn itself from something creaky and barely adequate into something radical and new; and I think London is a great place to begin.
Hence my first Ten Times Manifesto gig is now cancelled provisionally booked for 6pm, 10th August 2018 at the well-known Google Campus London, 4-5 Bonhill Street, London EC2A 4BX, which has space for 135 people.
The description runs:
After a lifetime writing software (including classic retro 1980s games such as Frak! and Zalaga), Nick Pelling created the Ten Times Manifesto to try to answer the question: how can we software developers do all the things we do but 10x better and/or 10x faster?
Expect a lot of good ideas, a lot of interaction, a lot of gratuitous movie references, and to go home afterwards with a lot to think about.
Core audience: anyone who develops software and wants to do what they do 10x better. 😉
When tickets become available (very shortly), they’ll be free – so anyone who wants to come along should move fast. 😉 I hope to see 135 of you there!
That is a brilliant goal you have there, Nick. A certain corporation whose name begins with ‘Micro’ and ends with ‘soft’ could do with some of that 10x code improvement. If only your talk were mandatory attendance for some on their team . . . say the ones responsible for forced ‘Feature’ updates that lately have a tendency to compromise drive partitions—corrupting RECOVERY volumes—repaying their collateral dam… err… I mean customers’ loyalty with bricked laptops.
lol – No, I’m not bitter. Even back in ‘95, the ‘Start Me Up’ advert did at least redact the Stone’s line: “You make a grown man cry.” 😉 Anyway, more power to you. Here’s wishing many more software developers realize your Manifesto and practice your philosophy in the best interest of the End User. Cheers!
What a dreary unimaginitive pro-mo for a symposium promoting new strategies to getting stuff done instantly or faster. What about FestoXManiX or ManiX-FestoX, something with savoir fare which the same ten times (XX) message, but in a less overbearing,big brotherly ordered manner. Good luck with the bash anyhow Nick; but don’t expect to see me in the hopefully full house Thames-side this time around.
Rob: thanks for the support + vote of confidence, much appreciated. 🙂
As to whether the kind of thing I’m aiming for would be good or bad for large corporations, we’ll have to see. I’d like to think what I’m aiming for will be good for everybody. 😉
John sanders: it’s my first gig with a whole heap of new ideas, so until it actually happens and people respond I won’t know whether I’m dreaming a solitary dream or glimpsing some kind of future. As the old adage goes, the best way to predict the future is to build it. 😉
I’ve worked with a lot of startups and have seen a lot of also-rans and I encounter the same problems year after year…
One of the biggest problems in the software industry is that corporations hire managers who don’t know anything about software development. The best managers I’ve met were developers themselves and had some concept of what was difficult and what was trivial and could “speak the language” of the programmers.
Another big problem is that those who hire software engineers assume they are also software designers and, in most cases, this is not true. Interface design is a special skill and only about 10% of programmers seem to have it, but you rarely see ads for “software designers” because the execs don’t have a clue that they are different skill sets and you need both.
A third problem is that many companies are reluctant to hire outside testers. I’ve worked with many companies who thought it was fine for the programmers to test their own software. They never once solicited the opinions of their potential user base, nor did they use any testers from their potential user base. Fixing the software before it goes to market is far easier (and far less costly) than trying to change things after it ships and it’s astounding how many companies think the people who coded the software (who often are not the same profile as the user base) are qualified to test it.
A fourth problem is that hiring is sometimes done entirely at the management level without consulting the software development staff. Many managers do not realize that there are specialties in programming and that developers are people with different temperaments suitable for doing different things. Using IQ tests to hire has been the rage for the last few years, but some good designers and debuggers are not the 2,000-lines-of-code high-IQ types. Some are methodical and thorough and equally intelligent in different ways. The goal is to put together a TEAM that includes different skill sets so that designing, coding, testing, and debugging are all represented.
A fifth problem is that the industry is moving more and more toward using “canned” routines but often those routines aren’t well-designed in the first place.
Most of these problems are quite fixable, but it takes a bit of common sense and foresight, awareness (and putting egos aside) to accomplish them.
Nick, I’m glad to see you care enough about software development to want to improve it. You have my heart-felt support.
Oh poor Nick, wrote software, ay ?
I threw in the towel when Visual C++ 6.0 kept messing up my resource file and XP kept rearranging my icons, then later wouldn’t let me delete a file even though I was administrator, and even after I booted up in (pause for laughter here) “SAFE mode”. C# is an insult to humanity by the way.
Eventually switched to Macs, that went well until Apple decided to become a world champion Snoopathon.
Productivity ? Nick, they used to measure it in (more laughter here) lines of code per day or week or month.
The error in the entire conception is to use 100 year old factory analogies and terminology in a technology in which a single line of software code may save months of work and thousands of dollars and bring a project closer to completion. So much nonsense written about it. Software is the instantiation of ideas. “Productivity” ? WHAT exactly is that ? 10 ideas per hour ?
Teams ? “Workgroups” ? It’s individuals, no matter what the external windows dressing, framework or pretend organizational structure, that get the work done.
Even management, threatened by the personal empowerment and liberation of the personal computer, has striven mightily to marginalize, deprecate and even disavow the so called “IT” staff. Former FBI directory Louis Freeh had a personal computer removed from his desktop. Later they lost 30 million dollars on a database system that didn’t quite work.
James Pannozzi: the problem is that software is getting bigger, fatter, harder to write, harder to deliver, and harder to test. I read recently that a typical car being built in 2018 has more than a million lines of code – and that’s without a hint of any of the self-driving autonomous vehicle stuff that’s being developed at the moment. And to make all this, we’re still relying on ways of working that date back to the 1970s in all the important senses. In my opinion, something really big has to change with this edifice, or else the whole software industry will just die a horrible death. It’s not hard to see that coming, unfortunately. 🙁
You are welcome, Nick. One last thing – admittedly a bit more serious than my personal laptop woes – and a top news story we woke up to here, across the pond. . .
Whatever you developers do, please don’t make 3D printer software run 10x faster – I may need the time to RUN! That’s not El Oh El. That’s me with my hands up. \_(ツ)_/
Nick,
Intriguing idea. I am also a software developer and well aware of the craft vs. science nature of programming as well as the problems many of which J.K. Petersen outlined so succinctly. Unfortunately, I can’t jump the puddle to be at your presentation. Is there (hopefully) going to be a video, transcript, article or whatnot coming as a result of the presentation?
Wish I could be there. Been my complaint for a long time (I wrote my first program, machine code, 1956, Bendix g15D.) Software seems to be always way behind hardware. That said, as you’re aware, “coding” is not context independent. A poor student can’t be just taught to “code” beyond a few fundamental statements. Even with a definite product in mind, nastiness creeps in (see the Mythical Man-Month.)
In the early ’90’s I determined that most,if not all, computer languages are culturally based, for example c&c C (or c++, written by a friend) with Pascal , written by Nicholas Wirth. If you’re not of the culture of the writer, uphill. It’s not called a computer language by accident. Simply stringing chunks of code together or putting code in “wrappers” (ugh!) will not alleviate the problems. There’s a lot more to this rant, but I’ve let it slide for a long time.
Last, but not least, do not forget Godel’s Theorem. There’s always a bug.
Best,
Don
Fred Brandes: I’m trying to work out the logistics of filming and/or streaming at the moment, will post again when I have an update on this. 🙂
Don: it’s a difficult problem to solve, up there with the Voynich Manuscript and what to really make of Presidential tweets. 🙂
It’s not too big a spoiler for the talk to say that I don’t see the biggest underlying problem as one of language or even culture: but more that we’re trying to empty a lake with a teaspoon. A shinier teaspoon ain’t gonna get it done any quicker. 😉
J.K.Petersen: yes, these are all problems that aren’t going away. My particular interest is in seeing if we can collectively turn software development into something we are proud of, rather than something that is perpetually bumping along the bottom. If we can do that, perhaps many of the other problems will cease to be important. 😉
I’m also a long-term software developer, and I would suggest that we’ve already passed through multiple of these 10x transformations, and are better off for it. (Just for an example, consider (non-mobile) browsers and their javascript consoles, and compare that to the BASIC interpreters in 80-s era 8-bits, and compare that in turn to the days of punch cards and line printers).
Jake: I heartily agree that programming languages have improved over time: but not by 10x in total since C in 1973 in terms of developer productivity, and I’d be somewhat surprised if you’d argue they’re the root of the current malaise with software. :-/
Here we go for the logo Nick: Three uppercase X letters in line with non spaced buttressed arms, to represent ten times ten (XXX), as well as forming your large centered M for Manifesto. You’ll note that a W is also conveniently created; so you might like use the M &W to interpose the additional message ” My Way ” (or the highway). Like it? It’s yours!….
NP: As you might have realised by now, I’m also in software. I’ve worked in a number of mundane places with differing approaches to how things should be done – from my early days at Motorola where everything was Six Sigma and CMMI, to roles related to Government where everything these days is about ITIL. I used to insist that all of these method(ologie)s just added process layers that complicated delivery – a one line change doesn’t necessarily need the same amount of scrutiny as a full system replacement (although I’ll concede that depends on what the line you’re adding/removing/modifying does), but even ITIL which proclaims to adapt with complexity adds unnecessary overhead in most implementations. Part of the problem is that the practitioners probably don’t really understand what they’re doing, so it becomes a system of arbitrary process. I like to think that with these sort of methodologies (as indeed with processes in general) they should serve as guidance dictating the ‘spirit of the law’ rather than as direction outlining the ‘letter of the law’.
These days I work on project that has sporadic process (if any at all – we mockingly call ‘FrAgile’ development, where a hallway conversation might be considered a requirement). Perhaps not surprisingly, it took me very little time to work out that the lack of process can be just as hindering as the process overkill. That the lack of organisation, planning and coordination means that often work is delivered in several different varieties by several different developers – and that even our version control, change management and job triage ‘processes’ fail to have any adequacy when they’re semi-optional (eg. a lot of work ends up happening on the main trunk and potentially clobbering stuff on branches).
Perhaps there is no silver bullet (I think someoen even wrote a book called that). Certainly we don’t seem to have learned the most basic Engineering principle: Demand exact requirements, and hold the Business to them (I think most organistaions’ implementation of ‘Agile’ makes the whole concept of requirement so fluid that it doesn’t exist – although, that said it’s probably important to recognise that some fluidity in requirements needs to be allowed/catered for). Too often we seem to develop stuff because *shiny*…and there’s loads of improvements I’ve made in different roles that were either snuck in slyly because we couldn’t make the business owner realise the shit their system was in, or have never been used because it turns out we never needed that bell or whistle.
But I think the 2 biggest hindrances I’ve encoutnered are:
1) The idea that software is some magic created by these developer mages who know secret stuff. I’m sure you’ve had as many ‘we just want a simple fix’ that you can’t explain to users isn’t all that simple, as the ‘I want you to revolutionise the world by…’ which ends up being a config change. There’s a genuine attitude that there is some dark art being applied, and I think many devs play on that (I’ve personally seen a lot of “Can’t help, I’m totally snowed under by XYZ”, when I know XYZ is a 5 minute job). I think also there are few adequate metrics as to the Magic Points in any given dev. I’ve seen people who seem to be incredibly competent in one language, but struggle with other similar languages (and I can’t explain why). I’ve also worked with many people whose code I simply struggled to trust (yes, I know, I’m a bit of an elitist and rate myself). Often even (self) nominated experts in a particular technology have struck me as making some peculiar decisions in their implementations. And if I, an insider, struggle to see/measure/realise the talent of my peers, then we can’t expect a manager (let alone a business) to be able to best harness the talents of their team (it doesn’t help that few technical people aspire to management, and those that do are 15years of technology behind by the time they get there).
2) Letting the business areas dictate tools. Many people compare Software Engineering to building/fixing a car. While I don’t particularly subscribe to that analogy, I will make the point that saying ‘We want you to fix this problem using only WebMethods (insert other technology if you like)” is akin to saying “We want you to change the tyre using only a hammer”. Rather than Enterprise Architecture being a way to unite disparate technologies (which are often used for a reason), it seems too often be a way to dictate exclusive technology use throughout an organisation. Understandably, the underlying hardware requires this to some degree, and there’s a lot to be said for standardising the toolsets across an organisation, however such an approach limits the ability of your coalface engineers of delivering the best product. I’m a C fanboi, and can cope with almost any ‘curly braced’ language (and indeed languages in other paradigms), however I can understand that there’s many reasons why C wouldn’t be the flavour of choice in many situations (these days I work a lot with Rails – mainly because people here like that things seem to be developed quickly, and don’t particularly care if processing is delayed (in fact there’s very much a “we’ll solve that problem by throwing more hardware at it if people complain enough” attitude (but sadly despite the complaint, people rarely want to pay for hardware)).
Sort of relevant to the discussion as well, is the elitism that exists within our ranks (and perhaps to a lesser degree the perception that we geeks aren’t good communicators). There are very talented dev’s who don’t trust anyone to touch their work (and to some degree rightly so), but then there’s layers underneath of less capable decs who believe they should have the same distrust. Worse, often the devs don’t like to share information, because they feel it might change their standing on the ladder. Then there’s also a layer of devs who seem to have gotten their job just because there was an oversupply of trolley boys at the super market. Often they have all sorts of ways of appearing busy, but when you dig into what they’re actually doing you struggle to find anything (in bad cases these people are deliberately shifted to work that isn’t expected to be delivered, or that can be weasled out of when it doesn’t deliver).
I remember a role where people wanted a system replaced. When we quizzed them as to why, they said “because it’s old” (it was originally written in the 90’s, but had had fundamental updates to underlying DB, OS etc). So we asked them what they wanted different…. “No, nothing, we want it exactly the same”. We struggled to convince them that if they want something exactly the same, we can simply charge them to leave the system as is (they even wanted the screeens exactly the same, lest they confuse their frontline users)…
Finally (and I think some of the startups get this right) you need to encourage people to do their own thing. At it’s most simple this is ‘toolsmithing’, but it also includes the development of genuine capability the business didn’t realise they want. Some of my greatest ‘masterpieces’ are non-commissioned work, usually that was created because I was lazy and wanted to do something faster so I have more time to do nothing. In one role, people used to come to me because they knew I could get them an answer quickly, but when I tried to encourage them to use my tools to do it themselves they didn’t want to hear about it (as far as they were concerned, I was solving problems manually, just faster than anyone else).
I think it’s admirable (and even theoretically achievable) to drive down development time and cost by a factor of 10. But I don’t think there’s an easy way to do it, and it will upset a lot of people (both within and outside the industry) along the way. Good Luck Nick. I think you’re going to need it…
milongal: thanks very much for sharing your experiences, much appreciated! While I too don’t think there’s ever any easy way of making significant improvements, a lot of what I’ll be talking about is that the nature of software development itself has radically shifted over the last 40+ years: somewhere along the line, the things software developers do in a day (and indeed the process hoops we all jump through) aren’t supported by the the languages and development infrastructure we use. We’ve all got so used to low productivity that we don’t notice how pervasive it hs become: but once in a while, it takes someone to stand up and point to the naked emperor parading down the street. 😉
Another problem that came to mind after I posted above, that shows up in some of the larger corporations, is one of compartmentalization.
These days, often a programmer is given a specific task without being given an overview of how the requested block of code fits into the overall picture. Heck, the very structure of the workspace (cubicles) is a physical reflection of this administrative model.
.
As long-standing as some of these problems are, I really believe that some of them can be fixed through education and awareness. Many software development decisions are made by MBAs who are leapfrogged into management without knowing anything about the development process.
It doesn’t matter how brilliant the staff is, if the people running the show have no developer skills and insist on micromanaging (or throttling) the software developers just to keep up the appearance of being in charge, the result is not going to be good.
So… one of the places where I think these problems can be mitigated is at the university level. It’s not enough for an MBA to take a semester of Cobol and think they have the skills to manage programmers and oversee a complex development cycle. A great deal of software development is invention, solving problems that have never been solved before (or even existed before), and flowcharts cannot anticipate every bend in the road. Expecting a simple path from box A to triangle B is inadequate to foster the creativity that is necessary to chart new territory and new territory doesn’t always mean something earth-shattering, sometimes there’s a better way to solve a simple algorithm like requesting user input in a way that is ergonomic and effective.
.
And, something I haven’t mentioned yet… not every programmer is a good programmer. Half of them are on the bottom half of the bell curve (sorry, guys, but it’s true).
I’m forced to use lousy software everyday. Three clicks when only one is needed, boxes blanked out after asking for user input when the next input will be almost the same, inadequate errror-checking, and humans being asked to do the work of the software.
Credit-card-number input is a classic example of BAD programming. It doesn’t MATTER whether the user add spaces or dashes (or nothing in between the numbers)! Parsing them out is something a bright 12-year-old with three weeks of programming instruction can do with Perl or any other language.
It’s EASY to make the software do the work, and yet these lazy coders allow millions and millions of users to input perfectly good data that the coders refuse to parse, then they show the user a cryptic error message and make them type in the number again. UNFORGIVEABLY sloppy and lazy programming and completely fixable. This kind of garbage code permeates the software industry. It takes 15 minutes to fix this particular problem, one that shouldn’t have existed in the first place, and yet look how often it continues to happen!
Multiply routines like this times billions of lines of code and bad software probably steals several years from the lifetime of every individual who has to use it on a regular basis.
J.K. Petersen: I’m not sure if you know that I have an MBA, but please be reassured that I work hard not to conform to the kind of idiotic stereotype you highlight. 😉
Nick, hopefully you can tell from my first message that I’m not talking about MBAs who actually understand software development (those are the ones who SHOULD be managing projects), I’m talking about the ones who know next to nothing about computers.
I did some contract work for a leading engineering firm involved in space technology and drones, and the managers in the block in which I was working (mostly MBAs) had no knowledge of coding. They didn’t even use computers. The computers on their desks (which were more powerful than those used by the development and media staff) were never turned on. They barely knew how to use a mobile phone.
Their expectations of what was easy and what was hard was reversed. The staff could “suggest” what resources they needed to do their jobs, but the managers always had final approval and often the answer was no. And it was almost impossible to convince them of the importance of testing (both internal and external) even though they are providers of precision technology. The staff was competent, as far as I could tell, but severely hampered by not having someone in management with programming knowledge to advocate for them and support them in their decisions.
Nick, I’d be interested to hear your talk, but like others I won’t be able to be there. (I did consider it for a fleeting second).
For me, creating software is like creating a building. There are so many different possibilities, from a garden shed (or two), via a palace to a nuclear power plant. Needs and rules are very different, so I guess you will delimit somehow the application(s).
Rene: I’ve just had an enjoyable evening chatting with Philip Neal, and he asked much the same question.
My answer is (perhaps necessarily) incomplete, but for me the issue is that traditional declarative programming languages assume that all you’re ever trying to do is bring out the ineffable declarativity within everything you type, when software development now encompasses twenty or more different activities, of which capturing Business Logic is but one.
I guess what I’m reaching towards is a world where the whole idea of programming languages ceases to be such a Golden Hammer, if that makes sense. 🙂
A former boss compared software development to building sandcastles. We know they will be washed away by the inevitable tide, yet we still strive to make them grand and beautiful.
I know people who have been hired as Web Developers who have no programming skills at all. They patch together sites using wizards in canned packages provided by Web hosts.
Since this form of “development” (I hate to use that term) is increasing, the problems with declarative-languages seems to be intermediary between a multi-paradigm approach (use whichever tool is most appropriate) and the use of whatever canned packages make it look “good enough”.
.
Another industry-wide problem is that large companies buy up small companies with good products by the thousands, and shelve the good software so it won’t compete with their not-so-good software.
Hi Nick,
my work happens to be related (among others) to an area of software development where the dominating criteria are:
– the consequences of a failure of the software
– the associated maximum allowed risk of such a failure.
This is clearly a very tiny area of the overall software industry, but here the process is completely dominated by the testing and validation.
Since cost and schedule remains an issue also in safety-of-life applications, there is clearly a need for optimisation of the processes. This then concentrates on the validation, and the impact of this on the code generation might be quite the opposite of what I think you are having in mind.
For these cases, well-defined straightforward programming languages are desirable. They tend to be found at the start of the alphabet, and I don’t mean Basic 🙂
I was just wondering if this particular area of applications is included in your talk.
Rene: A-languages and safety-critical systems offer excellent examples of how an ounce of code can sometimes require a sackful of validation. But even though the language use makes up only a small part of such work, this is the part that Computer Science focuses on.
Ultimately, all I’m proposing is that we should rebalance the scales, i.e. that we use tools and methodologies that support and accelerate the other 90% of what software developers do – plumb, test, integrate, deliver.
I wonder how we will ever escape the “Tower of Babel” problem as it applies to software. With so many programming languages, data formats, versioning/dependency constraints, and the ever-shifting zeitgeist of development frameworks, it doesn’t take much to reach an impasse of interoperability.
David Oranchak: perhaps we’re already there. 🙁 Which is why I’m trying to make a difference. 🙂
Nick:
Actually I think that sort of is what I’m saying. Thanks to things like improved abstractions and extensive standard libraries it’s no big achievement to do in 100 lines (or less) of a modern-ish language than one could do in 1000 lines (or more) of 1973-era C. For example, nobody needs to write their own sort anymore now that qsort is in libc, and at a slightly higher level if you’re writing c++ you had better not write your own string class or, god forbid, hash table without an extremely good reason.
I think we’re also doing a much better job on things like version control (git has pretty much won, but any modern vcs (hg, svn, etc) is much more powerful productivity-wise than older ones or not using one at all).
I’m not going to defend modern software management, but it’s not as if we’ve regressed.
JKPeterson’s got a bunch of posts here about bad programmers, but I think the worst of 2018 software is a lot better than the worst software of 1998, and that’s because of tool improvement (and stackoverflow).
Also re: David Oranchak’s post, I’m again going to suggest that we’re much better off today than we were in the past. Remember all those different word processor formats, and how nowadays it’s pretty much just docx with the occasional rtf? And the wildly divergent unix systems that made autotools a good idea? Swig and cython mean that if you have a c-like api you can get bindings in all sorts of other languages, there are a bunch of languages beyond java that run on the jvm, and if something’s sitting on port 80 you can talk to it in whatever language you like. (I don’t do npm/webdev, I know things are kind of messed up there, but the rest of my points stand).
J. Kesinger wrote: “JKPeterson’s got a bunch of posts here about bad programmers, but I think the worst of 2018 software is a lot better than the worst software of 1998, and that’s because of tool improvement (and stackoverflow).”
No, the majority of my comments were not about bad programmers.
Most of my comments were about programmers hampered by managers without software development knowledge or experience. This is the problem I see most often in both large and small companies. Managers need both administrative AND development expertise to bridge the gap between execs and the development teams. It doesn’t matter how talented the staff is if managers are interfering rather than advocating for their needs.
I also tried to emphasize the fact that programmers have different skill sets and management often doesn’t realize this and hires based on a mistaken assumption that all programmers do the same thing (which is not even remotely true).
And, something that happens constantly in the software industry… small companies with good software are bought out so the software can be shelved so it doesn’t compete with less-well-made software by bigger companies. Literally tens of thousands of products have been shelved, some of them of excellent quality.
JKP: When you say:
“And, something that happens constantly in the software industry… small companies with good software are bought out so the software can be shelved so it doesn’t compete with less-well-made software by bigger companies. Literally tens of thousands of products have been shelved, some of them of excellent quality.”
These kind of issues occur across all sectors not just the software industry. This is hardly the place to get political, but the more familar one is with private enterprise as a whole the more conscious one gets about how many aspects of it are inefficient and that actually the state which laid the foundation for so many of these technologies does quite a good job. From my experience I have come across numerous structural inefficiencies in businesses, especially large ones, as I am sure most people have. So privatisation is not always the panacea that is claimed relative to nationalisation; just take the american healthcare system as an example. Anyway this is somewhat off topic, but I felt the need to vent.
The problem can be described as out-of-control “accidental” complexity. I use accidental in the Fred Brooks (Mythical Man Month) sense of not being essential, though it is usually intentional. (One way it’s recently manifested itself is in the proliferation of “frameworks”, because the programming languages they run on top of aren’t good enough.) There are two different approaches to solving the problem: top-down and bottom-up, both are needed, and both are connected.
I don’t know any easy solution to the top-down problem. It results in monstrosities like the Universal Credit system, and software consultants and “architects” who design overly complex systems and then outsource their development to large armies of mediocre programmers. At a slightly smaller scale, some of the problems can be solved by programmers and managers being made to take responsibility for the software they develop, and if it’s a bug-ridden mess it should they who clean it up, while the developers and managers of stabler and more fit-for-purpose systems move on to more interesting things. Another idea is to hire smarter programmers, and one way to identify them is to find out how many different programming languages (and therefore modes of thinking) they’re comfortable programming in, and whether they program in their spare time.
Very few people involved in software development are aware of the subject’s history, or even care about it, and I come from from a very different programming culture which got a lot of things right (we had integrated development environments, hypertext, AI, and sophisticated 3d color graphics years before anyone else did), then ended up being sidelined before being airbrushed out of history altogether. So what do I think? What’s my bottom-up solution? There are a number of levels.
The hardware should be designed to run the programming language(s) the software is written in, like the Burroughs mainframes and MIT and Xerox Lisp Machines. This allows the compiler’s code generator to be a lot simpler. Keep the syntax simple. Many people are horrified by Lisp’s parentheses, but if you use a prettyprinting editor, you don’t even notice them. Lisp’s parser is a few very short functions.
What about the operating system? Dan Ingalls once said, “An operating system is a collection of things that don’t fit into a language. There shouldn’t be one.” Memory management? The garbage collector. File system? You don’t need files, you periodically update your image instead. You can safely shut your computer down by pulling the plug out of the wall, and it takes a fraction of a second to boot and you continue where you left off. Type declarations? Hindley-Milner type inference.
What should the language look like? Algorithms, ones that do complex things, can generally do many different things at the same time, independently. So the solution is more naturally expressed as a directed graph, with data flowing through it, than as a sequence of instructions reading and writing variables and files. If you express your problem graphically, type inference becomes more straightforward and you can dispense with variables (which are just proxies for memory addresses) entirely. Text was invented to record speech which is inherently sequential. Why are we still using it for programming?
And that’s what, for the past few years, with a couple of breaks while I studied the Voynich Manuscript, I’ve been working on. My web pages are currently unusable, but will return soon in a new place.
I used to know people – at Cumbernauld in Scotland, and at Manchester University, who were working on machines which could run, natively, languages just like that. They, too, have been forgotten about.
I’d like to write more but I’ve probably used up enough electronic vellum. Another time.
Donald: thanks for that – I have nothing but good things to say about your Full Metal Jacket, and I also appreciate the rationale and thinking behind it. 🙂
At the same time, what I’m reaching towards is neither top-down nor bottom-up, but rather ways of taking back control of the middle and then working outwards from there. So there’s plenty of room for everyone at the same time. 😉
There’s always a third way you don’t think of. I look forward to your talk.
The death of shrink-wrapped software (that’s what I developed until Internet piracy killed the business) has slowed a lot of innovation. Many of the new ideas and good algorithms came from this sector.
I do custom solutions now (programming and media), but it’s generally for companies that hide the technology. It gives them an edge in their corner of the market, so they’re not inclined to share it. This kind of innovation doesn’t filter out in the same way as ideas that are on everyone’s desktop or PDA.
I’m supportive of open-source software, it helps stave off big-corp hegemony, but I rarely see the same kinds of innovation that were characteristic of shrink-wrapped products. I don’t know if this is because the monetary incentive isn’t there or because software-by-committee is always a different from “boutique” development, but something special seems to have been lost (or at least buried).
“Many people are horrified by Lisp’s parentheses…”
Haha, yes! I’ve noticed that. I’ve always liked Lisp (and Prolog). Each language has its place. I think of languages as items in a toolbox… grab the one that’s best for the job.
I’ve just today heard back from Google Campus London:
“Space at Campus is extremely popular and we receive a high volume of event applications. With this in mind, we are unfortunately not able to host you this time.”
I checked their event calendar again just now and the 6pm-7pm slot this Friday is still showing as free. So I can honestly say I have no idea why my application was turned down, and please accept my apologies to everyone for having to cancel at such ridiculously short notice. 🙁 🙁 🙁 🙁
Or if anyone has a good idea for other places I could book at short notice…?
Nick: Sorry to hear about that! It sounded like a very interesting talk; though I doubt you have ever given a boring talk, unless compelled to.
I am very sorry to hear it, Nick.
Not only is it an interesting topic, but I think it would have attracted an equally interesting crowd. Hopefully you can work something out.
Substitute that abominable ‘manifest’ word to something less provocative Nick; prospectus or proposition should pass muster for your follow-up booking.
john sanders: if people are so afraid of the power of individual words, something has gone badly wrong. 🙁
All too true Nick; but its all about perceptions these days and manifesto is perceived to be an order rather than a persuader.
john sanders: I guess the underlying problem is that I have a strong sense of mission with it, and calling it “the 10x prospectus/proposition” sounds to me too much like “the 10x it-would-be-nice-if-you-could-find-sufficient-time-to-consider-this-proposal-sorry-for-taking-up-so-much-of-your-time-already-I’ll-see-myself-out-thanks”. :-/
Nick: I guess the Cluetrain Manifesto was well received by the online business world back in ’99 (1999) and I’ve a feeling it’s aggressive general proposal is still fondly recalled. So stick to your guns man and don’t forget the old adage ‘the will to win and pluck to fight etc..’
Nick: Setting aside your recent disapointment; my own somewhat related vexation lies with the frustrations encounted on this ortherwise terrific site, due to the non-contemporaneus set up which, because of time differences, doesn’t allow for post vetting within a reasonable time frame. This situation is in need of improvement so that someone in Australia or the USA for onstance, might keep up with the information flow from non UK sources over a full 24 hour period. With repect js.