Cloud computing - The Findability blog

Each year CIO Sweden conducts a trend survey among Sweden’s CIOs. They also host an annual event where they discuss the results and the CIOs from some of Sweden’s largest companies talk about their vision. On February 6 I attended this year’s CIO Trends event at Münchenbryggeriet in Stockholm.

The main conclusion from this year’s survey is that compared to last year not that many things have changed. However, one interesting change this year was that last year’s most important trend, cloud and cloud solutions, had been kicked down by Mobility. Mobility as in easiness to move around not only in the office but also in large scales around the world. Information should always be on your fingertip no matter the device or connection. The Cloud is still a hot topic and focus on that is still high among companies. I guess henceforth we will see more of a combination of the two where you use cloud to create more mobility.

Fun fact of the day from the CIOs of Sweden: The most common CIO in Sweden is Male (84%), around 45-49 years old (33%) and don’t like shopping (2%).

//Ludvig Aldrin still Sweden’s youngest CIO (CIO’s under 30, 1%)

In June 2011, I attended the Berlin Buzzwords conference. The main theme of the conference was undoubtedly the current paradigm shift in distributed processing, driven by the major success of Hadoop. Doug Cutting – founder of Apache projects such as Lucene, Nutch and Hadoop – held one of the keynotes. He focused on what he recognized as the new foundations for this paradigm shift:

– Commodity hardware
– Sequential file access
– Sharding
– Automated, high level reliability
– Open source

Distributed processing is done fairly well with Hadoop. Distributed search on the other hand is more or less limited to sharding and/or replicating the index. The downside of sharding is that you perform the same search on multiple servers and then need to combine the results. Due to the nature of algorithms in search such as tf/idf, tasks like ranking results suffers. Andrzej Białecki (another frequent Lucene committer) held a presentation on this topic, and his view can be summarized as: Use local search as long as you can, distribute only when the cost of local search limitations outweighs the cost of distributed search.

The setup of automated replication and sharding, with help from Zookeeper in the Solr Cloud project, is a major step in the right direction but the question on how to properly combine search results from different nodes still remains. One thing is sure though, there is a lot of interesting work being done in this area.