The Dangerous Powers of Algorithms

Author: Arvind Padmanabhan

An opinion about the good and the bad


When Guglielmo Marconi arrived in England in the 1890s, he brought with him the idea that information could be wirelessly sent across a distance. For a world still attached to wired telegraphy, this was a revolutionary idea. Marconi gave successful demonstrations to astonished audiences. Not many understood the principles. The equipment he carried were soon called “magic boxes”, except that there was no magic in them. They were simply cleverly constructed transmitters and receivers.

Today there’s a vaguely parallel development happening with data. Though lot of data is open, the algorithms that make use of them are closed. With algorithms, often employed for profit and hence proprietary, the “magic” is purposely hidden from those who wish to see them. Indeed, magicians thrive on the principle that tricks never be revealed. With the growing use (and misuse) of data in every sphere of life, we are becoming slowly aware of the important role that algorithms play in our lives. There’s a growing concern that algorithms may be working more against us than for us.

Continue reading “The Dangerous Powers of Algorithms”


Why Big Data Doesn’t Work for Me

Author: Arvind Padmanabhan

Examples of Big Data going wrong


My earlier articles on big data presented quite a rosy picture. Big data is supposed to do this, supposed to do that and eventually make this world a better place. Any new technology has its period of hype and expectations. There are always teething troubles. Basic things don’t work as expected. Technology needs time to mature.

Strangely, very few even realize that there are problems with big data. Data scientists and engineers dealing with big data know that there are improvements to be made but they are not fully aware how ineffectively big data analytics work today. They give a solution based on certain assumptions but funnily they don’t actually know what a particular user needs. My own personal experience will serve to vindicate these points.

Continue reading “Why Big Data Doesn’t Work for Me”