Review our cookies information for more details Special report: Managing information A different game Information is transforming traditional businesses Feb 25th 2010 | from the print edition * * IN 1879 James Ritty, a saloon-keeper in Dayton, Ohio, received a patent for a wooden contraption that he dubbed the “incorruptible cashier”. With a set of buttons and a loud bell, the device, sold by National Cash Register (NCR), was little more than a simple adding machine. Yet as an early form of managing information flows in American business the cash register had a huge impact.
It not only reduced pilferage by alerting the shopkeeper when the till was opened; by recording every transaction, it also provided an instant overview of what was happening in the business. Sales data remain one of a company’s most important assets. In 2004 Wal-Mart peered into its mammoth databases and noticed that before a hurricane struck, there was a run on flashlights and batteries, as might be expected; but also on Pop-Tarts, a sugary American breakfast snack. On reflection it is clear that the snack would be a handy thing to eat in a blackout, but the retailer would not have thought to stock up on it before a storm.
The company whose system crunched Wal-Mart’s numbers was none other than NCR and its data-warehousing unit, Teradata, now an independent firm. A few years ago such technologies, called “business intelligence”, were available only to the world’s biggest companies. But as the price of computing and storage has fallen and the software systems have got better and cheaper, the technology has moved into the mainstream. Companies are collecting more data than ever before. In the past they were kept in different systems that were unable to talk to each other, such as finance, human resources or customer management.
Now the systems are being linked, and companies are using data-mining techniques to get a complete picture of their operations—“a single version of the truth”, as the industry likes to call it. That allows firms to operate more efficiently, pick out trends and improve their forecasting. In this special report * Data, data everywhere * All too much * »A different game * Clicking for gold * The open society * Show me * Needle in a haystack * New rules for big data * Handling the cornucopia Sources & acknowledgementsReprints Related topics * China * Nestle * IBM * Royal Shakespeare Company * Walmart
Consider Cablecom, a Swiss telecoms operator. It has reduced customer defections from one-fifth of subscribers a year to under 5% by crunching its numbers. Its software spotted that although customer defections peaked in the 13th month, the decision to leave was made much earlier, around the ninth month (as indicated by things like the number of calls to customer support services). So Cablecom offered certain customers special deals seven months into their subscription and reaped the rewards. Agony and torture Such data-mining has a dubious reputation. “Torture the data long enough and they will confess to anything,” statisticians quip.
But it has become far more effective as more companies have started to use the technology. Best Buy, a retailer, found that 7% of its customers accounted for 43% of its sales, so it reorganised its stores to concentrate on those customers’ needs. Airline yield management improved because analytical techniques uncovered the best predictor that a passenger would actually catch a flight he had booked: that he had ordered a vegetarian meal. The IT industry is piling into business intelligence, seeing it as a natural successor of services such as accountancy and computing in the first and second half of the 20th century respectively.
Accenture, PricewaterhouseCoopers, IBM and SAP are investing heavily in their consulting practices. Technology vendors such as Oracle, Informatica, TIBCO, SAS and EMC have benefited. IBM believes business intelligence will be a pillar of its growth as sensors are used to manage things from a city’s traffic flow to a patient’s blood flow. It has invested $12 billion in the past four years and is opening six analytics centres with 4,000 employees worldwide. Analytics—performing statistical operations for forecasting or uncovering correlations such as between Pop-Tarts and hurricanes—can have a big pay-off.
In Britain the Royal Shakespeare Company (RSC) sifted through seven years of sales data for a marketing campaign that increased regular visitors by 70%. By examining more than 2m transaction records, the RSC discovered a lot more about its best customers: not just income, but things like occupation and family status, which allowed it to target its marketing more precisely. That was of crucial importance, says the RSC’s Mary Butlin, because it substantially boosted membership as well as fund-raising revenue. Yet making the most of data is not easy. The first step is to improve the accuracy of the information.
Nestle, for example, sells more than 100,000 products in 200 countries, using 550,000 suppliers, but it was not using its huge buying power effectively because its databases were a mess. On examination, it found that of its 9m records of vendors, customers and materials around half were obsolete or duplicated, and of the remainder about one-third were inaccurate or incomplete. The name of a vendor might be abbreviated in one record but spelled out in another, leading to double-counting. Plainer vanilla Over the past ten years Nestle has been overhauling its IT system, using SAP software, and improving the quality of its data.
This enabled the firm to become more efficient, says Chris Johnson, who led the initiative. For just one ingredient, vanilla, its American operation was able to reduce the number of specifications and use fewer suppliers, saving $30m a year. Overall, such operational improvements save more than $1 billion annually. Nestle is not alone in having problems with its database. Most CIOs admit that their data are of poor quality. In a study by IBM half the managers quizzed did not trust the information on which they had to base decisions. Many say that the technology meant to make sense of it often just produces more data.
Instead of finding a needle in the haystack, they are making more hay. Still, as analytical techniques become more widespread, business decisions will increasingly be made, or at least corroborated, on the basis of computer algorithms rather than individual hunches. This creates a need for managers who are comfortable with data, but statistics courses in business schools are not popular. Many new business insights come from “dead data”: stored information about past transactions that are examined to reveal hidden correlations. But now companies are increasingly moving to analysing real-time information flows.
Wal-Mart is a good example. The retailer operates 8,400 stores worldwide, has more than 2m employees and handles over 200m customer transactions each week. Its revenue last year, around $400 billion, is more than the GDP of many entire countries. The sheer scale of the data is a challenge, admits Rollin Ford, the CIO at Wal-Mart’s headquarters in Bentonville, Arkansas. “We keep a healthy paranoia. ” Not a sparrow falls Wal-Mart’s inventory-management system, called Retail Link, enables suppliers to see the exact number of their products on every shelf of every store at that precise moment.
The system shows the rate of sales by the hour, by the day, over the past year and more. Begun in the 1990s, Retail Link gives suppliers a complete overview of when and how their products are selling, and with what other products in the shopping cart. This lets suppliers manage their stocks better. The technology enabled Wal-Mart to change the business model of retailing. In some cases it leaves stock management in the hands of its suppliers and does not take ownership of the products until the moment they are sold. This allows it to shed inventory risk and reduce its costs.
In essence, the shelves in its shops are a highly efficiently managed depot. Another company that capitalises on real-time information flows is Li & Fung, one of the world’s biggest supply-chain operators. Founded in Guangzhou in southern China a century ago, it does not own any factories or equipment but orchestrates a network of 12,000 suppliers in 40 countries, sourcing goods for brands ranging from Kate Spade to Walt Disney. Its turnover in 2008 was $14 billion. Li ; Fung used to deal with its clients mostly by phone and fax, with e-mail counting as high technology.
But thanks to a new web-services platform, its processes have speeded up. Orders flow through a web portal and bids can be solicited from pre-qualified suppliers. Agents now audit factories in real time with hand-held computers. Clients are able to monitor the details of every stage of an order, from the initial production run to shipping. One of the most important technologies has turned out to be videoconferencing. It allows buyers and manufacturers to examine the colour of a material or the stitching on a garment. “Before, we weren’t able to send a 500MB image—we’d post a DVD.
Now we can stream it to show vendors in our offices. With real-time images we can make changes quicker,” says Manuel Fernandez, Li ; Fung’s chief technology officer. Data flowing through its network soared from 100 gigabytes a day only 18 months ago to 1 terabyte. The information system also allows Li & Fung to look across its operations to identify trends. In southern China, for instance, a shortage of workers and new legislation raised labour costs, so production moved north. “We saw that before it actually happened,” says Mr Fernandez.
The company also got advance warning of the economic crisis, and later the recovery, from retailers’ orders before these trends became apparent. Investment analysts use country information provided by Li ; Fung to gain insights into macroeconomic patterns. Now that they are able to process information flows in real time, organisations are collecting more data than ever. One use for such information is to forecast when machines will break down. This hardly ever happens out of the blue: there are usually warning signs such as noise, vibration or heat. Capturing such data enables firms to act before a breakdown.
Similarly, the use of “predictive analytics” on the basis of large data sets may transform health care. Dr Carolyn McGregor of the University of Ontario, working with IBM, conducts research to spot potentially fatal infections in premature babies. The system monitors subtle changes in seven streams of real-time data, such as respiration, heart rate and blood pressure. The electrocardiogram alone generates 1,000 readings per second. This kind of information is turned out by all medical equipment, but it used to be recorded on paper and examined perhaps once an hour.
By feeding the data into a computer, Dr McGregor has been able to detect the onset of an infection before obvious symptoms emerge. “You can’t see it with the naked eye, but a computer can,” she says. Open sesame Two technology trends are helping to fuel these new uses of data: cloud computing and open-source software. Cloud computing—in which the internet is used as a platform to collect, store and process data—allows businesses to lease computing power as and when they need it, rather than having to buy expensive equipment.
Amazon, Google and Microsoft are the most prominent firms to make their massive computing infrastructure available to clients. As more corporate functions, such as human resources or sales, are managed over a network, companies can see patterns across the whole of the business and share their information more easily. A free programming language called R lets companies examine and present big data sets, and free software called Hadoop now allows ordinary PCs to analyse huge quantities of data that previously required a supercomputer. It does this by parcelling out the tasks to numerous computers at once. This saves time and money.
For example, the New York Times a few years ago used cloud computing and Hadoop to convert over 400,000 scanned images from its archives, from 1851 to 1922. By harnessing the power of hundreds of computers, it was able to do the job in 36 hours. Visa, a credit-card company, in a recent trial with Hadoop crunched two years of test records, or 73 billion transactions, amounting to 36 terabytes of data. The processing time fell from one month with traditional methods to a mere 13 minutes. It is a striking successor of Ritty’s incorruptible cashier for a data-driven age. from the print edition | Special report Recommend 140 * * * Submit to reddit * inShare2 * View all comments (4) Related items TOPIC: China » * Recommended economics writing: Link exchange * Trade: Mexico rising * The Economist: Digital highlights, November 24th 2012 TOPIC: Nestle » * Consumer goods in Africa: A continent goes shopping * Schumpeter: Pretty profitable parrots * Nestle buys Pfizer Nutrition: Feeding little emperors TOPIC: IBM » * Schumpeter: Taking the long view * IBM’s mainframes: Old dog, new tricks * Phase-change memory: Altered states TOPIC: Royal Shakespeare Company » * William Shakespeare: A digital reinvention Culture: Going for gold * Green architecture: The retrofit revolution More related topics: * Walmart Want more? Subscribe to The Economist and get the week’s most relevant news and analysis. * Print edition X Feb 27th 2010 Feb 20th 2010 Feb 13th 2010 Feb 6th 2010 * Next in The world this week X Politics this week * Next in The world this week X Business this week * Next in The world this week X KAL’s cartoon * Next in Leaders X Technology The data deluge Businesses, governments and society are only starting to tap its vast potential * Next in Leaders X Argentina and the Falklands The beef in Buenos Aires
The Kirchners could have more oil if they stopped bullying Argentine business * Next in Leaders X Japan’s frustrating politics Nagasaki fallout Japan’s prime minister, Yukio Hatoyama, should jettison his Svengali, Ichiro Ozawa * Next in Leaders X India Ending the red terror It is time India got serious about the Maoist insurgency in its eastern states * Next in Leaders X Genetically modified food Attack of the really quite likeable tomatoes The success of genetically modified crops provides opportunities to win over their critics * Next in Letters X Letters On Spain, al-Qaeda, Yemen, torture, Britain, juries, stereotypes, Benjamin Disraeli Next in Briefing X Argentina under the Kirchners Socialism for foes, capitalism for friends While some private businesses in Argentina have faced harassment or even nationalisation, others… * Next in Briefing X The first family’s businesses Welcome to the Hotel Kirchner Such a lovely little earner * Next in United States X Health reform Seizing the reins, at long last After leaving Congress in charge for too long, Barack Obama unveils his own plan * Next in United States X Mitt Romney and the Republicans Fired up, ready to go Mitt Romney takes centre-stage * Next in United States X The administration’s economists
Grading the dismal scientists How good is the Council of Economic Advisers? * Next in United States X The economy Back to the crash The American economy has just had its worst decade since the 1930s * Next in United States X Arkansas politics Democrats beware A spirited scramble for suddenly open Democratic seats * Next in United States X Schools and testing The finger of suspicion Is too much weight given to testing? * Next in United States X California’s prison-guards’ union Fading are the peacemakers One of California’s most powerful political forces may have peaked * Next in United States X America’s children
Protecting the weakest The recession may hurt America’s vulnerable children * Next in United States X Lexington Is Barack Obama tough enough? Conservatives call him too weak to be a warrior. Tell that to the Taliban * Next in The Americas X Corruption in Brazil The money trail Many corruption scandals stem from the high cost of politics, and unrealistically tight… * Next in The Americas X Presidential ambitions in Peru Political satire Jaime Bayly’s breath of fresh air * Next in The Americas X Latin American summitry In ever-closer union, divided we stand * Next in The Americas X Canada’s Mohawks Get out of our canoe
When a Canadian is not a Canadian * Next in Asia X Tackling Japan’s bureaucracy Floundering in the foggy fortress The DPJ is finding that it needs to befriend its bureaucrats, as well as bash them * Next in Asia X India’s Naxalite insurgency Not a dinner party India’s Maoist guerrillas carry out two slaughters, then offer a truce * Next in Asia X Western aims in Afghanistan Played for fools Hamid Karzai’s shenanigans make the going even harder for NATO * Next in Asia X Migrant workers in Thailand Inhospitality Life gets harder for Thailand’s guest-workers * Next in Asia X China’s National People’s Congress
Democracy in action Making sure that China’s supreme legislative body is toothless * Next in Asia X Animal welfare in China Off the menu The right to eat cats and dogs is under threat * Next in Asia X Banyan The mother of all dictatorships To understand North Korea, look not to Confucius or the Soviet Union, but to fascist 1930s Japan * Next in Asia X Clarification: Maratha * Next in Middle East and Africa X Israel’s controversial intelligence service Does Mossad really make Israel safer? In the wake of the assassination of a Hamas leader in Dubai, presumably by Mossad, the agency’s… * Next in Middle East and Africa
X Israeli spies in Lebanon Not such a success A round-up of Israeli spies * Next in Middle East and Africa X Senegal’s politics Statuesque or grotesque? An outsize statue symbolises the defects of the president and his family * Next in Middle East and Africa X South Africa’s economy Steady as she goes A budget that gives little to the left * Next in Middle East and Africa X Somalia’s civil war Jihadists on the march The strongest Islamist militia is now formally linked to al-Qaeda * Next in Middle East and Africa X Niger’s coup It seems popular, so far The African Union tut-tuts but the people appear to welcome a coup Next in Middle East and Africa X Nigeria’s president A sudden return There’s still a vacuum * Next in Middle East and Africa X Correction: Najib Balala * Next in Europe X Ukraine’s new president Yanukovich’s mixed blessing A triumphant Viktor Yanukovich is inaugurated in Kiev, but his political problems have only just… * Next in Europe X Germany’s fractious government Westerwelle’s woes A liberal at odds with some of his coalition partners * Next in Europe X Latvia and Greece Baltic thaw, Aegean freeze Latvia’s economic free fall has halted, and it may now do better than Greece * Next in Europe X Turkey’s coup plotters
Lies and whispers More arrests stoke the battle between the army and the government * Next in Europe X The Dutch government falls Wild things The far right promises to do disturbingly well on June 9th * Next in Europe X France’s Socialist Party Fresh troubles A scandal in the south-west shows up the metropolitan party’s weaknesses * Next in Europe X Charlemagne Europe’s bear problem The trouble with the European Union’s attempts to woo Russia * Next in Britain X Britain’s prime minister Hero, villain or victim of the global age? Gordon Brown’s pitch for a fourth Labour term—and his critique of the Tories Next in Britain X The Conservatives’ mini-malaise Tory blues The opposition is failing to capitalise on the government’s troubles * Next in Britain X Media and the law Publish, perish, protest Bad news for dodgy journalism—and for libel tourists * Next in Britain X Pensions and tax If it says ARF, then it’s a dog Pensions rules get complex again * Next in Britain X Helped suicide The latest chapter New rules on helping those who wish to end their lives but are unable to do so * Next in Britain X Construction jitters Survival tactics Building firms are struggling to emerge from the recession * Next in Britain
X Bagehot All too human How much does a prime minister’s character matter? * Next in Britain X Internship * Next in International X Sanctions on Iran And the price of nuclear power? America is rallying its friends to concentrate minds in the Islamic Republic * Next in International X A poll on trust What’s good for General Motors A new pattern in opinions about bureaucrats, business and charity * Next in Special report X Data, data everywhere Information has gone from scarce to superabundant. That brings huge new benefits, says Kenneth… * Next in Special report X All too much Monstrous amounts of data Next in Special report X Clicking for gold How internet companies profit from data on the web * Next in Special report X The open society Governments are letting in the light * Next in Special report X Show me New ways of visualising data * Next in Special report X Needle in a haystack The uses of information about information * Next in Special report X New rules for big data Regulators are having to rethink their brief * Next in Special report X Handling the cornucopia The best way to deal with all that information is to use machines. But they need watching * Next in Business X Recruitment firms Joining the queue
The recession has accelerated big changes for firms that help people find jobs * Next in Business X The spread of GM crops Taking root The developing world embraces a controversial technology * Next in Business X The boom in printing on demand Just press print New technology promises to prolong the life of the book * Next in Business X Hype about fuel cells Flower power A clean-tech start-up generates lots of excitement and a little electricity * Next in Business X A boardroom row at Repsol Adding fuel A policy shift in Spain heralds more upheaval at its biggest oil firm * Next in Business X Toyota’s overstretched supply chain
The machine that ran too hot The woes of the world’s biggest carmaker are a warning for rivals * Next in Business X Schumpeter The emperor’s clothes Like other bosses, media moguls are recovering their poise. But that’s no reason to start making… * Next in Briefing X Reviving Royal Bank of Scotland Scots on the rocks What really went wrong at RBS? And how can it be put right? * Next in Finance and economics X The balance of economic power East or famine Asia’s economic weight in the world has risen, but by less than commonly assumed * Next in Finance and economics X Emerging-market sovereign debt Risk redefined
The new problem with Asian sovereign debt—scarcity * Next in Finance and economics X Secondary buy-outs Circular logic Private-equity companies look to each other to solve their problems * Next in Finance and economics X Interest-rate risk Surf’s up Banks’ next big problem appears on the horizon * Next in Finance and economics X Buttonwood The very long view For investors, much depends on when you put your money in * Next in Finance and economics X Chinese banks Hole sale Capital calls by Chinese financial institutions elicit questions * Next in Finance and economics X Short-selling rules Shackling the scapegoats
American regulators approve long-awaited restrictions on short-selling * Next in Finance and economics X Economics focus Low definition Trustbusters want to put less emphasis on market definition when assessing mergers * Next in Finance and economics X Correction: Financial risk * Next in Science and technology X Climate and combustion Fired up This year’s meeting of the American Association for the Advancement of Science looked at, among… * Next in Science and technology X How siestas help memory Sleepy heads Researchers say an afternoon nap prepares the brain to learn * Next in Science and technology X
Looking for ET Signs of life As the search for alien life turns 50, its practitioners find new methods * Next in Science and technology X Nuclear forensics A weighty matter How to analyse smuggled uranium * Next in Books and arts X China’s roads A voyage of discovery A reporter who explored China’s bigger and lesser roads and found treasure * Next in Books and arts X How East Timor became Timor-Leste A country’s agonising birth An authoritative account of Timor-Leste’s birth * Next in Books and arts X New York low life Bottoms up Essays on New York by St Clair McKelway, taken from the New Yorker * Next in Books and arts
X University education in America Professionalising the professor The difficulties of an American doctoral student * Next in Books and arts X A biography of Arthur Koestler Intellectual fireworks A serial fornicator with a powerful, paradoxical intellect * Next in Books and arts X A Japanese silversmith Making waves Pounding flat pieces of silver into beautiful vessels * Next in Books and arts X Old men of the theatre The two Peters A couple of productions that make a compelling case against ageism * Next in Obituary X Alexander Haig Alexander Meigs Haig, soldier and public servant, died on February 20th, aged 85 Next in Economic and financial indicators X Overview * Next in Economic and financial indicators X Output, prices and jobs * Next in Economic and financial indicators X The Economist commodity-price index * Next in Economic and financial indicators X FDIC-insured “problem” institutions * Next in Economic and financial indicators X Trade, exchange rates, budget balances and interest rates * Next in Economic and financial indicators X Markets * Next in Economic and financial indicators X Wall Street bonuses * Print edition X Feb 27th 2010 Mar 6th 2010 Mar 13th 2010 Mar 20th 2010 From the print edition Feb 27th 2010 Comment (4) * Print * E-mail * Reprints ; permissions * Most popular * Recommended * Commented Recommended * 1Mexico and the United StatesThe rise of Mexico * 2International: The lottery of life * 3Atheists and Islam: No God, not even Allah * 4European economy guide: Polarised prospects * 5Egypt: Going up in flames Commented * 1Atheists and IslamNo God, not even Allah * 2France and the euro: The time-bomb at the heart of Europe * 3Secession and elections: Let’s stay together * 4Higher education: Not what it used to be * 5Economist debate: Opening Latest blog posts – All times are GMT
Christmas countdown: The 2012 Daily chart Advent calendar Graphic detail – Dec 1st, 06:54 Environmental policy: Trent Lott outs himself as the owner of a titchy,… Lexington’s notebook – Dec 1st, 03:45 Mexico’s new cabinet: Out with the old, in with the… old Americas view – Dec 1st, 03:06 Recommended economics writing: Link exchange Free exchange – Nov 30th, 22:11 The Securities and Exchange Commission: Merger talks Schumpeter – Nov 30th, 21:50 Defending Grover Norquist: Nice try, John Democracy in America – Nov 30th, 20:45 Czech politics: The Czech Republic’s first presidential debate Eastern approaches – Nov 30th, 20:23
More from our blogs » Products & events Stay informed today and every day Get e-mail newsletters Subscribe to The Economist’s free e-mail newsletters and alerts. Follow The Economist on Twitter Subscribe to The Economist’s latest article postings on Twitter Follow The Economist on Facebook See a selection of The Economist’s articles, events, topical videos and debates on Facebook. * About The Economist * Media directory * Advertising info * Staff books * Career opportunities * Subscribe * Contact us * Site index * [+] Site Feedback * Copyright © The Economist Newspaper Limited 2012.
All rights reserved. * Accessibility * Privacy policy * Cookies info * Terms of use * Help * Log in * Register * Subscribe * Digital ; mobile * Events * Topics A-Z * Newsletters * Jobs * * * * Top of Form Bottom of Form * World politics * ————————————————- United States * ————————————————- Britain * ————————————————- Europe * ————————————————- China * ————————————————- Asia * ————————————————- Americas ————————————————- Middle East & Africa * Business & finance * ————————————————- All Business & finance * ————————————————- Which MBA? * ————————————————- Business Books Quarterly * Economics * ————————————————- All Economics * ————————————————- Economics by invitation * ————————————————- Economics A-Z * ————————————————- Markets & data Science & technology * ————————————————- All Science & technology * ————————————————- Technology Quarterly * Culture * ————————————————- All Culture * ————————————————- More Intelligent Life * ————————————————- Style guide * ————————————————- The Economist Quiz * ————————————————- Book reviews * Blogs * ————————————————- Latest blog posts ————————————————- Feast and famine * ————————————————- Analects * ————————————————- Free exchange * ————————————————- Americas view * ————————————————- Game theory * ————————————————- Babbage * ————————————————- Graphic detail * ————————————————- Banyan * ————————————————- Gulliver * ————————————————- Baobab ————————————————- Johnson * ————————————————- Blighty * ————————————————- Lexington’s notebook * ————————————————- Buttonwood’s notebook * ————————————————- Newsbook * ————————————————- Cassandra * ————————————————- Pomegranate * ————————————————- Charlemagne * ————————————————- Prospero * ————————————————-
Democracy in America * ————————————————- Schumpeter * ————————————————- Eastern approaches * Debate * ————————————————- Economist debates * ————————————————- What the world thinks * ————————————————- Economics by invitation * ————————————————- Letters to the editor * ————————————————- Ideas arena: Women & work * The World in 2013 * Multimedia * ————————————————-
World * ————————————————- Business & economics * ————————————————- Science & technology * ————————————————- Culture * ————————————————- Events * ————————————————- The Economist in audio * Print edition * Current issue * Previous issues * Special reports * Politics this week * Business this week * Leaders * KAL’s cartoon * Obituary This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.
Review our cookies information for more details Special report: Managing information A different game Information is transforming traditional businesses Feb 25th 2010 | from the print edition * * IN 1879 James Ritty, a saloon-keeper in Dayton, Ohio, received a patent for a wooden contraption that he dubbed the “incorruptible cashier”. With a set of buttons and a loud bell, the device, sold by National Cash Register (NCR), was little more than a simple adding machine. Yet as an early form of managing information flows in American business the cash register had a huge impact.
It not only reduced pilferage by alerting the shopkeeper when the till was opened; by recording every transaction, it also provided an instant overview of what was happening in the business. Sales data remain one of a company’s most important assets. In 2004 Wal-Mart peered into its mammoth databases and noticed that before a hurricane struck, there was a run on flashlights and batteries, as might be expected; but also on Pop-Tarts, a sugary American breakfast snack. On reflection it is clear that the snack would be a handy thing to eat in a blackout, but the retailer would not have thought to stock up on it before a storm.
The company whose system crunched Wal-Mart’s numbers was none other than NCR and its data-warehousing unit, Teradata, now an independent firm. A few years ago such technologies, called “business intelligence”, were available only to the world’s biggest companies. But as the price of computing and storage has fallen and the software systems have got better and cheaper, the technology has moved into the mainstream. Companies are collecting more data than ever before. In the past they were kept in different systems that were unable to talk to each other, such as finance, human resources or customer management.
Now the systems are being linked, and companies are using data-mining techniques to get a complete picture of their operations—“a single version of the truth”, as the industry likes to call it. That allows firms to operate more efficiently, pick out trends and improve their forecasting. In this special report * Data, data everywhere * All too much * »A different game * Clicking for gold * The open society * Show me * Needle in a haystack * New rules for big data * Handling the cornucopia Sources & acknowledgementsReprints Related topics * China * Nestle * IBM * Royal Shakespeare Company * Walmart
Consider Cablecom, a Swiss telecoms operator. It has reduced customer defections from one-fifth of subscribers a year to under 5% by crunching its numbers. Its software spotted that although customer defections peaked in the 13th month, the decision to leave was made much earlier, around the ninth month (as indicated by things like the number of calls to customer support services). So Cablecom offered certain customers special deals seven months into their subscription and reaped the rewards. Agony and torture Such data-mining has a dubious reputation. “Torture the data long enough and they will confess to anything,” statisticians quip.
But it has become far more effective as more companies have started to use the technology. Best Buy, a retailer, found that 7% of its customers accounted for 43% of its sales, so it reorganised its stores to concentrate on those customers’ needs. Airline yield management improved because analytical techniques uncovered the best predictor that a passenger would actually catch a flight he had booked: that he had ordered a vegetarian meal. The IT industry is piling into business intelligence, seeing it as a natural successor of services such as accountancy and computing in the first and second half of the 20th century respectively.
Accenture, PricewaterhouseCoopers, IBM and SAP are investing heavily in their consulting practices. Technology vendors such as Oracle, Informatica, TIBCO, SAS and EMC have benefited. IBM believes business intelligence will be a pillar of its growth as sensors are used to manage things from a city’s traffic flow to a patient’s blood flow. It has invested $12 billion in the past four years and is opening six analytics centres with 4,000 employees worldwide. Analytics—performing statistical operations for forecasting or uncovering correlations such as between Pop-Tarts and hurricanes—can have a big pay-off.
In Britain the Royal Shakespeare Company (RSC) sifted through seven years of sales data for a marketing campaign that increased regular visitors by 70%. By examining more than 2m transaction records, the RSC discovered a lot more about its best customers: not just income, but things like occupation and family status, which allowed it to target its marketing more precisely. That was of crucial importance, says the RSC’s Mary Butlin, because it substantially boosted membership as well as fund-raising revenue. Yet making the most of data is not easy. The first step is to improve the accuracy of the information.
Nestle, for example, sells more than 100,000 products in 200 countries, using 550,000 suppliers, but it was not using its huge buying power effectively because its databases were a mess. On examination, it found that of its 9m records of vendors, customers and materials around half were obsolete or duplicated, and of the remainder about one-third were inaccurate or incomplete. The name of a vendor might be abbreviated in one record but spelled out in another, leading to double-counting. Plainer vanilla Over the past ten years Nestle has been overhauling its IT system, using SAP software, and improving the quality of its data.
This enabled the firm to become more efficient, says Chris Johnson, who led the initiative. For just one ingredient, vanilla, its American operation was able to reduce the number of specifications and use fewer suppliers, saving $30m a year. Overall, such operational improvements save more than $1 billion annually. Nestle is not alone in having problems with its database. Most CIOs admit that their data are of poor quality. In a study by IBM half the managers quizzed did not trust the information on which they had to base decisions. Many say that the technology meant to make sense of it often just produces more data.
Instead of finding a needle in the haystack, they are making more hay. Still, as analytical techniques become more widespread, business decisions will increasingly be made, or at least corroborated, on the basis of computer algorithms rather than individual hunches. This creates a need for managers who are comfortable with data, but statistics courses in business schools are not popular. Many new business insights come from “dead data”: stored information about past transactions that are examined to reveal hidden correlations. But now companies are increasingly moving to analysing real-time information flows.
Wal-Mart is a good example. The retailer operates 8,400 stores worldwide, has more than 2m employees and handles over 200m customer transactions each week. Its revenue last year, around $400 billion, is more than the GDP of many entire countries. The sheer scale of the data is a challenge, admits Rollin Ford, the CIO at Wal-Mart’s headquarters in Bentonville, Arkansas. “We keep a healthy paranoia. ” Not a sparrow falls Wal-Mart’s inventory-management system, called Retail Link, enables suppliers to see the exact number of their products on every shelf of every store at that precise moment.
The system shows the rate of sales by the hour, by the day, over the past year and more. Begun in the 1990s, Retail Link gives suppliers a complete overview of when and how their products are selling, and with what other products in the shopping cart. This lets suppliers manage their stocks better. The technology enabled Wal-Mart to change the business model of retailing. In some cases it leaves stock management in the hands of its suppliers and does not take ownership of the products until the moment they are sold. This allows it to shed inventory risk and reduce its costs.
In essence, the shelves in its shops are a highly efficiently managed depot. Another company that capitalises on real-time information flows is Li & Fung, one of the world’s biggest supply-chain operators. Founded in Guangzhou in southern China a century ago, it does not own any factories or equipment but orchestrates a network of 12,000 suppliers in 40 countries, sourcing goods for brands ranging from Kate Spade to Walt Disney. Its turnover in 2008 was $14 billion. Li ; Fung used to deal with its clients mostly by phone and fax, with e-mail counting as high technology.
But thanks to a new web-services platform, its processes have speeded up. Orders flow through a web portal and bids can be solicited from pre-qualified suppliers. Agents now audit factories in real time with hand-held computers. Clients are able to monitor the details of every stage of an order, from the initial production run to shipping. One of the most important technologies has turned out to be videoconferencing. It allows buyers and manufacturers to examine the colour of a material or the stitching on a garment. “Before, we weren’t able to send a 500MB image—we’d post a DVD.
Now we can stream it to show vendors in our offices. With real-time images we can make changes quicker,” says Manuel Fernandez, Li ; Fung’s chief technology officer. Data flowing through its network soared from 100 gigabytes a day only 18 months ago to 1 terabyte. The information system also allows Li & Fung to look across its operations to identify trends. In southern China, for instance, a shortage of workers and new legislation raised labour costs, so production moved north. “We saw that before it actually happened,” says Mr Fernandez.
The company also got advance warning of the economic crisis, and later the recovery, from retailers’ orders before these trends became apparent. Investment analysts use country information provided by Li ; Fung to gain insights into macroeconomic patterns. Now that they are able to process information flows in real time, organisations are collecting more data than ever. One use for such information is to forecast when machines will break down. This hardly ever happens out of the blue: there are usually warning signs such as noise, vibration or heat. Capturing such data enables firms to act before a breakdown.
Similarly, the use of “predictive analytics” on the basis of large data sets may transform health care. Dr Carolyn McGregor of the University of Ontario, working with IBM, conducts research to spot potentially fatal infections in premature babies. The system monitors subtle changes in seven streams of real-time data, such as respiration, heart rate and blood pressure. The electrocardiogram alone generates 1,000 readings per second. This kind of information is turned out by all medical equipment, but it used to be recorded on paper and examined perhaps once an hour.
By feeding the data into a computer, Dr McGregor has been able to detect the onset of an infection before obvious symptoms emerge. “You can’t see it with the naked eye, but a computer can,” she says. Open sesame Two technology trends are helping to fuel these new uses of data: cloud computing and open-source software. Cloud computing—in which the internet is used as a platform to collect, store and process data—allows businesses to lease computing power as and when they need it, rather than having to buy expensive equipment.
Amazon, Google and Microsoft are the most prominent firms to make their massive computing infrastructure available to clients. As more corporate functions, such as human resources or sales, are managed over a network, companies can see patterns across the whole of the business and share their information more easily. A free programming language called R lets companies examine and present big data sets, and free software called Hadoop now allows ordinary PCs to analyse huge quantities of data that previously required a supercomputer. It does this by parcelling out the tasks to numerous computers at once.
This saves time and money. For example, the New York Times a few years ago used cloud computing and Hadoop to convert over 400,000 scanned images from its archives, from 1851 to 1922. By harnessing the power of hundreds of computers, it was able to do the job in 36 hours. Visa, a credit-card company, in a recent trial with Hadoop crunched two years of test records, or 73 billion transactions, amounting to 36 terabytes of data. The processing time fell from one month with traditional methods to a mere 13 minutes. It is a striking successor of Ritty’s incorruptible cashier for a data-driven age. rom the print edition | Special report * Recommend 140 * * * Submit to reddit * inShare2 * View all comments (4) Related items TOPIC: China » * Recommended economics writing: Link exchange * Trade: Mexico rising * The Economist: Digital highlights, November 24th 2012 TOPIC: Nestle » * Consumer goods in Africa: A continent goes shopping * Schumpeter: Pretty profitable parrots * Nestle buys Pfizer Nutrition: Feeding little emperors TOPIC: IBM » * Schumpeter: Taking the long view * IBM’s mainframes: Old dog, new tricks * Phase-change memory: Altered states TOPIC: Royal Shakespeare Company » William Shakespeare: A digital reinvention * Culture: Going for gold * Green architecture: The retrofit revolution More related topics: * Walmart Want more? Subscribe to The Economist and get the week’s most relevant news and analysis. * Print edition X Feb 27th 2010 Feb 20th 2010 Feb 13th 2010 Feb 6th 2010 * Next in The world this week X Politics this week * Next in The world this week X Business this week * Next in The world this week X KAL’s cartoon * Next in Leaders X Technology The data deluge Businesses, governments and society are only starting to tap its vast potential * Next in Leaders
X Argentina and the Falklands The beef in Buenos Aires The Kirchners could have more oil if they stopped bullying Argentine business * Next in Leaders X Japan’s frustrating politics Nagasaki fallout Japan’s prime minister, Yukio Hatoyama, should jettison his Svengali, Ichiro Ozawa * Next in Leaders X India Ending the red terror It is time India got serious about the Maoist insurgency in its eastern states * Next in Leaders X Genetically modified food Attack of the really quite likeable tomatoes The success of genetically modified crops provides opportunities to win over their critics Next in Letters X Letters On Spain, al-Qaeda, Yemen, torture, Britain, juries, stereotypes, Benjamin Disraeli * Next in Briefing X Argentina under the Kirchners Socialism for foes, capitalism for friends While some private businesses in Argentina have faced harassment or even nationalisation, others… * Next in Briefing X The first family’s businesses Welcome to the Hotel Kirchner Such a lovely little earner * Next in United States X Health reform Seizing the reins, at long last After leaving Congress in charge for too long, Barack Obama unveils his own plan * Next in United States
X Mitt Romney and the Republicans Fired up, ready to go Mitt Romney takes centre-stage * Next in United States X The administration’s economists Grading the dismal scientists How good is the Council of Economic Advisers? * Next in United States X The economy Back to the crash The American economy has just had its worst decade since the 1930s * Next in United States X Arkansas politics Democrats beware A spirited scramble for suddenly open Democratic seats * Next in United States X Schools and testing The finger of suspicion Is too much weight given to testing? * Next in United States X
California’s prison-guards’ union Fading are the peacemakers One of California’s most powerful political forces may have peaked * Next in United States X America’s children Protecting the weakest The recession may hurt America’s vulnerable children * Next in United States X Lexington Is Barack Obama tough enough? Conservatives call him too weak to be a warrior. Tell that to the Taliban * Next in The Americas X Corruption in Brazil The money trail Many corruption scandals stem from the high cost of politics, and unrealistically tight… * Next in The Americas X Presidential ambitions in Peru
Political satire Jaime Bayly’s breath of fresh air * Next in The Americas X Latin American summitry In ever-closer union, divided we stand * Next in The Americas X Canada’s Mohawks Get out of our canoe When a Canadian is not a Canadian * Next in Asia X Tackling Japan’s bureaucracy Floundering in the foggy fortress The DPJ is finding that it needs to befriend its bureaucrats, as well as bash them * Next in Asia X India’s Naxalite insurgency Not a dinner party India’s Maoist guerrillas carry out two slaughters, then offer a truce * Next in Asia X Western aims in Afghanistan
Played for fools Hamid Karzai’s shenanigans make the going even harder for NATO * Next in Asia X Migrant workers in Thailand Inhospitality Life gets harder for Thailand’s guest-workers * Next in Asia X China’s National People’s Congress Democracy in action Making sure that China’s supreme legislative body is toothless * Next in Asia X Animal welfare in China Off the menu The right to eat cats and dogs is under threat * Next in Asia X Banyan The mother of all dictatorships To understand North Korea, look not to Confucius or the Soviet Union, but to fascist 1930s Japan * Next in Asia X
