Predicting the Future (Part 1)

In the last post I outlined the Research Roadmap Process, the first phase of the process is predicting the future. That is a huge task, in fact, some may say it is impossible, particularly in these trying times. As I write this most of the world is coping with the spread of the COVID-19 virus. The world today looks very different to what it did just 1 month ago.

Can you Predict the Future?

These is a huge degree of scepticism whenever you start talking about predicting the future. Your right to initially argue that it is impossible. I mean after all, if we could predict the future we’d have predicted and coped better with COVID-19, or on a more positive note predicted the winning lottery numbers.

I’d like to introduce you to four people, all of whom have tried to predict the future.

Robert FitzRoy

Captain Robert FitzRoy was the Royal Navy Captain who sailed Charles Darwin aboard HMS Beagle on his famous voyage to The Galápagos Islands. But that wasn’t the peak of his career. FitzRoy became fascinated by the weather and, in an effort to try to save fellow sailors, he attempted to predict it. In the 1800s this idea seemed simply impossible; how could you possibly predict the future? FitzRoy faced such a huge pushback. This ridicule and scepticism from the media and the scientific community drove FitzRoy to create his own word - “Forecast”. He argued that this word better portrayed the work he was doing:

“Prophecies and predictions they are not, the term forecast is strictly applicable to such an opinion as is the result of scientific combination and calculation.”

FitzRoy went on to found the UK’s Met Office, become Vice-Admiral and eventually Governor of New Zealand. Perhaps his lasting legacy for us in the realm of software and technology research is that he showed that predicting the future, even with uncertainty, can be done.

William Gibson

In 1984 William Gibson released a Science Fiction book called Neuromancer. In this book Gibson outlined a future in which we access “the net”. On “the net” Gibson predicted that there would be large commercial corporations, computer servers, firewalls and indeed hackers. Just as FitzRoy invented his own word so did Gibson. He called this new place on “the net” Cyberspace.

Marc Porat

In 1978 Marc Porat was Program Director at the Aspen Institute, an international non-profit think tank. It was while here that Porat wrote what he referred to as his “red book”. This book contained a vision of the future that Porat had come up with. In this vision of the future Porat envisioned a world where we all walked around with phones in our pockets. He even created sketches for what the device would look like. It would have a large screen covering the entire face of it and the screen would react to your touch.

Porat also detailed what you would do with the device. He argued that you would receive electronic messages which would just drop from a computer and land on your phone. Just as Gibson had created his own word so did Porat. In trying to describe how these servers in the sky which dropped messages to mobile phones would work. Porat referred to them as being a “cloud” of computers.

If you would like to see an image of the phone that Porat had in his red book, then please head over to the General Magic Movie website. Indeed if you’d like to learn more about the movie you can stream it online, it’s a fantastic watch, it details the startup company Porat ran, called General Magic.

As these three figures have shown, predicting the future is possible. The complete accuracy of the prediction however, well, that can be debated. Before we start thinking about how we can predict the future we need to think about how far into the future we want to look. FitzRoy only looked 24 - 48 hours into the future. We’d need to look further ahead than that.

How far ahead do I have to look?

Deciding how far into the future to try to predict is difficult but there are several key drivers from your organisation which can help you try to decide what that time frame will be. We can work backwards to determine what this time frame should be.

The Technology Transfer Latency

Imagine that right now you had a fantastic detailed plan for the future and the perfect future product. You hand this future product and plan over to a development team. It will take the development team time to convert that product into something that can be sold. We can call this the Technology Transfer Latency. This latency differs depending on the organisation. For most large companies this can be about two years: the first year validating the future product can be sold and the next year spent creating it and polishing it to the standards required in the marketplace.

The Time Spend Researching

Once you’ve understood your company’s technology transfer latency, the next question is how long do you need to conduct your research to validate your vision of the future. This can be dependent on available budgets and organisational structures. Your internal team may have its own project length for instance. Collaborative research however often comes with its own fixed set of research timelines.

Collaborative Research

Both the USA and the EU sponsor commercial collaborative research. Both organisations provide a prebuilt method of allowing both academic and commercial organisations to work together. The frameworks are so well structured that they often encourage competing organisations to collaborate, whereas perhaps in the past they wouldn’t have. I once took part in an EU sponsored research project called CloudWave , it was about how to adapt software for better execution in the cloud. I represented Intel, working to make cloud applications better on Intel’s x86 processor platform. At the same time, I worked with IBM who were attempting to do the same for the PowerPC processor platform.

In providing the collaborative research frameworks, both the EU and the USA provide project duration guidelines. In the case of the EU and its Horizon 2020 programme this is roughly three years, broken down into three stages. These are understanding the problem, developing and evaluating possible solutions and, in the last phase, creating a working prototype which addresses the issues. In this case the research period is about three years.

Now imagine that you have a vision of the future, by working backwards we have determined we need this to extend about five years in the future (the Technology Transfer Latency of two years and the research period of three years). But we still need time to create the vision of the future in the first place. It can take up to a year to understand the business, the customers, the competitors, the value chain, the technical challenges and potential solutions. This gives us a period of about six years.

The amount of time you spend trying to predict the future, the technology realisation delay and the amount of time you are able to spend doing the research all vary. Different organisations have different views on what these should be and to a large extent this is dictated by available budget. However, as a rough rule of thumb 5 - 7 years is the most valuable. When you try to predict further into the future the number of variables you need to try to consider jumps massively and the risk of simply getting it completely wrong is very high. Conversely when you consider shorter time frames of two to three years ahead there is very little scope for new innovation as you are basically working on validating the next version of the product.

Methods for Predicting the Future

Now we know how far into the future we need to predict the next question is “how?”. There are a number of different approaches which can be used to try to make educated guesses on the future. They appear to fall into two main categories: a wide-breadth or large trend observation and a deep-seeing, product or industry specific observation.

Watching the Waves at the Beach

I am going to use a quick analogy here to explain wide-breadth and deep-seeing. Imagine that you are standing on a sandy beach. The beach has a few rocks off to the left-hand side, and some tidal pools, scooped out sand caused by the motion of the waves. You are standing watching the waves come in to the beach and you can see the patterns that they form on the shore. The strong wind from the sea is responsible for the major wave pattern, however, the local rocks bounce the waves off each other and the waves interact, causing the tidal pools, which further effect the waves. The result at the shore side is a pattern which appears random but it isn’t. It is however extremely complex.

By understanding the strength of the wind, we can make educated predictions on the size of the waves we might see at the shore. However, it is almost impossible to understand all the effects of the rocks and tidal pools. We might be able to predict the effect of the waves on a small area of the rocks but not for the entire stretch of the beach we can see before us.

The large sweeping trends, both social and technological, which cross the globe are akin to the wind whipping up waves; this is the wide-breath approach. In contrast, studying the effects of only a small number of rocks gives us a deep-seeing approach. The best predictions are of course when these are combined. But doing that is a huge and complex but not an impossible task which we will cover in the next post.

From the Earth to the Moon

Remember I mentioned we were going to talk about four people? Well we’ve only covered three. In 1865 Jules Verne wrote the book “From the Earth to the Moon”, in which he depicted a giant cannon which could fire a projectile all the way from the Earth and land it on the moon. This is of course impossible but the length of the cannon and its location were not. You see Verne had, prior to writing the book, done a considerable amount of research. He had calculated the speed at which a projectile would need to leave earth’s atmosphere. Based on this he deduced the size of the cannon required. He also looked hard at where in the world you would want to place the cannon. This search was a combination of politics; which nation would have the funds and the drive to launch something to moon, and it was also science based; placing a launch site closer to the equator allowed for the reduced thickness of the earth’s atmosphere at that latitude and allowed the projectile to utilise the earth’s rotation. Which location did Verne pick? Florida. In fact an amazing number of his calculations were correct and informed his story so much that folks have been comparing the Apollo moon missions to his book for a while.

The best Science Fiction and the best interpretations of the future are based, just as FitzRoy said, on “the result of scientific combination and calculation”. What Verne had done was identify the core mathematics and physics behind an attempt at a moon landing and then worked backward toward a story.

The Next Blog Post

This post has already become longer than anticipated. In the next post I am going to outline how ‘breadth first’ approaches work. But in the meantime, think about your organisation and industry. Can you identify any core motivations that direct and drive the work you do? Let me know. you can reach me on Twitter as @mcwoods.