Posted on

Our love affair with big data

Twenty years ago this summer, I was writing up my PhD thesis, about how to interpret and visualise big data from bridges, except we didn’t call it that back then. We just called it large data sets which really isn’t as sexy by half. Telling someone that you work in big data or should that be BIG DATA? To me, it feels like you are telling everyone how important it is Harry Enfield style: loadsadata like I’m the one with the big swinging data.

Funny because when data mining was all the rage, no one said it in the same way that they do BIG DATA. It sounded more like the ploughing that furrow approach that doing a PhD requires and we are all underground mining away with applications like Tesco’s clubcard, coming up with new ways to entice people to buy more things. Now, we are putting AI and BIG DATA together, well that has to be more meaningful, and more important right? We must be at least explaining the mysteries of the universe.

Or, are we?

Err no. We cannot discover brand new information using the information we have. We can make links and combine the information we capture, and eventually rethink what we are capturing with our new insights. By linking data together we see where we should monitor new things, rather like the time John Snow plotted outbreaks of cholera on a map and saw that they were near the same water pumps and then, a long time after that everyone realised that it wasn’t the fog killing people. But, it cannot tell you something brand spanking new, off the chain, never heard of before theories, if you are still matching your new data to old stories.

My PhD used models, i.e., different algorithms, which interpreted data in various ways to get the best fit to the data. Simply put, one of my users would be watching the measurements come in and then depending on what the data looked like would choose the most likely explanation. By changing various constraints the user could explore a small set of other but similar potential explanations. Nonetheless, we were mapping likely explanations onto the data before us. Unless the data was completely strange looking and then it was either ditched, or the odd outlier was deleted and then the set was interpreted to fit. Tailored, nipped and tucked.

And, that is how it works. I’ve said before that we are always trying to understand how the world works and our place in it. I’ve talked about how we reason and how we choose our stories. Nothing really changes.

It can be summed up perfectly by the time I was doing English Literature ‘A’ level. We were doing Beckett Waiting for Godot and I remember saying something that was just not what you say about Beckett, you have to say stuff about existentialist angst and the futility of it all and trot out the accepted interpretations of the play to pass the exam. Basically, you have to be a parrot. I think the teacher must have gotten frustrated and said that if someone says something different they are either a genius or an idiot and either way, it’s just unacceptable. I can’t help but think that is the way we run society. It takes ages to convince people of a new way of doing things unless someone gets rich quick and then everyone wants to know and have a go.

When I lectured AI I would start with the idea of readily transferable knowledge. I would point out the idea that it takes years to raise and educate a human until they are fully functional in the workplace, 16 years old at least and then years more to get the experience they need until they are wise and expert. Imagine if you could replicate that in an instant. Then, I would pause and the students would say: Ahhhhh.

Older and wiser now I am not so sure that instant knowledge is a good thing. It depends on who coded it up.

I listen to my girls in awe as they view the world so interestingly and refreshingly and I know that they are human and are able to reason universally unlike a machine. Sometimes they say the most inspiring things. Things I’ve forgotten I used to think naturally which bring me back to myself and how I used to be, before I was told how things are supposed to be.

They also talk sometimes a lot of nonsense as they don’t have the necessary information, or wisdom, or constraints. And, often when I explain that something can’t be done and they ask why not, alas, I am like my Eng Lit teacher saying: That’s just not a thing. It takes a few days before I will come back with a new answer of, yes, it could be a thing, we could do this. And, I can’t help but think that’s how we feel about big data.

We think that with enough data and enough tech we will discover something new and inspiring or fabulous and it will fulfil our yearnings and longings and bring us back to ourselves, the selves we already know, but have forgotten because we are conditioned by society.

I said to my husband that big data is our attempt to use technology to come back to ourselves and he said that he would definitely run that past the Machine Learning Department at work. But it’s true even there, all that searching is to find out a new USP or a competitive edge to make us stand out, to feel special, to win, but within reason as it has to fit within how things already work. And, here’s the rub: Any bit of AI has to have specific instructions to know what it is looking for in its big data. It won’t just randomly discover some brand new thing. Remember genius or idiot? It doesn’t have a universal reasoning system like my girls. It is just not possible in a computer. And, won’t be for a long time, if ever.

So perhaps big data is like having an affair.

Esther Perez’s book on infidelity says that most people don’t have an affair to have sex. They have an affair to rediscover themselves. They are seen through new eyes and appreciated once more in what feels like new and interesting ways. When we get that yearning for feeling that we matter, that we are important, that we mean something, we are sexy and alive, we often look for the answers outside ourselves and people who have affairs do so by looking to other people. Other people who look at us with love, like we are hot and the centre of the whole universe. Like the sun which everything revolves around.

However, unless someone is very self-unaware and even then, in the throws of a most exciting affair, most people wouldn’t discover something previously completely unheard of about themselves. What happens is that something the person takes for granted, or indeed may be taken for granted at home, may have a spotlight shone on it by their new love and then they appreciate themselves better and feel all loved up. They feel love. The love they’ve always had within them. The love within their love.

So, the question is: Is big data the spouse or your new love? Will you take it on home afterwards? Or, will you keep collecting data? And, most importantly, is it telling your whole story? The right story of you. The answer depends on who coded it. If it is a white middle class male, and/or a nerd, then unless you are one of those, you may be searching forever, ‘cos that is not where your story lies. Your story may not be in the model selection. It may not have even be collected in the data. You might not fit at all. You could have been deleted as an outlier.

My PhD data had to follow the laws of physics about how bridges stay up, so that is a big constraint. So there wasn’t a great deal of quantum physics mystical room for interpretation and lovin’ and collective unconsciousness. But, even without strict physical constraints, a lot of the time we hash out the same old stories because they fit in the same old world, same old society, same old ways of how we operate even though it is not based on the laws of physics. It’s based on economics.

Take web design, when we all first came online everyone was super creative because no one had any expectations of how a website should look. Nowadays, all sites look the same as they are designed by grid, yawn, ‘cos we are managing expectations and herding people through our site to do the things we want them to do, we are desperate not to lose them. Desperate for their money.

And then we have intuition. AI can’t tell if your data or indeed your interpretation is useful, only a human can intuit that, unless you get AI checking your AI but then you still need a human somewhere to tell all the AI what to do.

Like my blog, according to Jetpack I was having on average six people a day visiting and yet, I am often offered money to link to other sites, and so if I don’t get visitors why would people want to give me money to buy my links? Peculiar.

In the how to make a shed load of money online circles you have to build community and work really hard and share data of a specific size and shape and content. I didn’t do any of that, because I just want to write nice things which interest me. I am not selling anything, including my links. I guess this site is my big data, about me, and I do the interpreting and the love affairs and the linking to me, me, me.

Eventually, I deleted Jetpack because it was making my site grind really slowly and I couldn’t write my blogs without it timing out. So, following Jetpack’s lead as they themselves use Google analytics, I installed that instead, but I really couldn’t be bothered with the work it would take to configure it. Why make the UI so hard to understand and use? It assumes so much prior knowledge and not about how web stats should work, but about how it’s own system works without making that obvious in hte design. So, a couple of months ago I went back to the raw data on my cpanel to really see. Guess what? As you can see from above, I get 1.2 million hits and nearly 300,000 visitors a year. I am just amazed and thrilled and I feel love. I figured no one cared what I had to say, because that is a story I’ve long believed and been told, and I didn’t bother writing a new story or even checking my facts. Whoah!

Afterwards I wondered, who are these 300,000 people reading my site? My husband reckons nerdy people like him. And, then he added: Like you. I sat there for ages going I’m not a nerd. I am a woman, which honestly is a whole blog series right there. My husband said that I am sexist. He could be right. I’ll have to check the data first.

That said, if I have a lot of hits and not much community then that sounds right doesn’t? Nerds like reading not talking, and that’s what I do too. I rarely comment on someone’s site. I guess I am a nerd, but I don’t feel that I fit the nerd model. The one I was sold all those years ago. And, as much as we all like to resist labels that’s how we interpret the world – our social identities, how we categorise others, how we try to understand patterns. Sometimes, and this is sooo telling, when people can’t put me in a box, they will ask me what my husband does (rude!). And, moreover that is how AI interprets its big data. Oh yes! It cannot work any other way.

The best bit about looking at my data with fresh eyes was the key search terms, as Google when it did the personalisation thing (you know serving up what you’ve already seen or very similar pages based on what you’ve seen), also did away with search terms, but other browsers still collect it. Obviously Google hasn’t stopped collecting, it has just stopped sharing it, as my guess it wants to keep it for itself as information is power. But as The Stonekeeper says in Smallfoot: What are you going to do with that power?

Here are my top three search terms non-Googlified:

  1. Sex positions to lose weight
  2. Full human parts of a man in dialog
  3. Semiotics and storytelling

I hope these people found the things they were looking for, and I hope they read my take on semiotics and storytelling. However, say I was googling sex positions to lose weight, would I leave a comment? No! I mean I don’t leave a comment when I am googling recipes or anything. Imagine: hi, yes that reverse cowgirl position really did it for me. I lost four lbs in a week.

And, it is right up there with our top motivation of being seen and heard, often we are told by society that we are too fat, too slutty, too loud, too experienced, too inexperienced, too, too, too. So, again we must bend ourselves out of shape to get into the shape we think we will get loved for. We must fit the story, the model, the expectations.

So, who wouldn’t want big data to come along and tell us what is fabulous and the best thing about us? Alas, it cannot and will not do that until we learn to do that for ourselves. We need to light our own fires.

When I think of data, I love the story Sims creator Will Wright, told on Masterclass, I have lost all my notes and the classnotes so there are no links. Gah. Anyway, he said how he used all the data that someone collected in the 1960’s to understand and map out daily timetables for people’s Sims, as people had access to building whole worlds, but very few people wanted to do that, most people just wanted to replicate the world they live in. Imagine, you have the power to create worlds and you just want to make what you already know and see, you want to make a world in which you go to work and hate your job, just like your real world, you don’t want to make something new.

Technology is an extension of us, it reflects our concerns and motivations, and until we have new ones, and new stories, technology can only propagate what already exists, just on a much bigger scale. Is that what we want?

Personally speaking, I want technology to reflect all the love we can possibly feel and then some. At the very least, that’s where I would start.

Posted on

Digital Culture

pic borrowed from maltatimes.com

When we think of culture, we often think of art galleries and museums, places where curators decide which works of art should be preserved and presented to future generations. Treasuring artefacts and storytelling has been a way of recording history since civilisation began. The cultural gatekeepers, who judge what cultural achievement is and which artefacts or stories should be preserved, have been until now few in number, which makes their choices political ones.

However, culture is much bigger than art and cultural achievement. Culture is learnt, not inherited and yet it influences our biology, our behaviour, our individuality. It is often used interchangeably with nation, race, ethnicity, identity and community, and so it is not surprising that in 1952, anthropologists, Kroeber and Kluckhohn compiled a list of 164 definitions of culture.

We talk about culture in politics, education and the work place, particularly how organisations have different management cultures such as Amazon’s bruising culture and, the idea of a global culture with global citizens.

Before 1995, the Internet belonged more or less to the culture of academia, until Tim Berners-Lee created a client-server information system called the World Wide Web and ran it over the Internet. Berners-Lee’s goal was to give universal access to a large universe of documents.  He didn’t realise how the world would change and we would have another culture to consider: Digital Culture.

Digitisation and a cultural shift

Many arts and cultural organisations have turned to technology to help their archiving and preservation work, which in turn has led to a number of changes in these organisations and how they use technology.

They use technology to:

  • Automate processes such as ticket sales and fundraising.
  • Understand audience engagement with their exhibitions using data analysis.
  • Promote and reach new audiences with social media.

Digitising works of art allows easy reproduction and distribution to an audience worldwide, whilst technology reduces the overheads of staffing, print materials, etc., so that organisations can lower costs. Digital installations accompany exhibits and augmented reality apps like Aurasma, add an extra dimension and experience to an exhibit.

Digitisation and access to the Internet has led to a shift in cultural gatekeeping and opened the door to normal people creating too. Anyone in the world can create and contribute. It is no longer a small world where only the select few can say what is good and worth treasuring. And the use of technology creates new types of digital art, to add to the culture of art.

Self-publishing is a perfect example of this approach. Writers no longer have to wait for the gatekeepers of publishing to condone their work and present it to an audience. Instead, anyone with access to the Internet can publish their work electronically and on paper for a price, cheap or otherwise, to attract an audience. They no longer need to wait for someone else’s established approval.

News media and television journalism have been instrumental in shaping our collective memory for much of the twentieth century but now, thanks to twitter and facebook, no longer. There are many others alongside the gatekeepers of the media who influence where we focus our attention. Anyone with access to the Internet and a story to tell access new platforms to reach people who were once only accessed by the media. Indeed, the media often curate tweets and other social media posts to record public opinion when there is breaking news.

The Internet compresses time and space so that the night shift is always covered because it is the day shift somewhere else in the world. Banks operate these hours. When one member of a team goes to bed in London, they know that someone in New York will be there well into the night, and once night falls in New York, there will be someone else working away in Hong Kong.

This flexible workforce and creation of intangible products such as databases, knowledge, or apps is known as a weightless economy or knowledge economy. Once something is made it can be reproduced and distributed at a low cost; infinitely.

Theoretically, this should ensure there is enough for everyone much more easily than it would be when sharing out physical goods and resources. Often though, the opposite is seen. This is because of the ‘superstar effect’. Consumers prefer to buy famous or branded knowledge goods e.g, ebooks, songs, movies or apps. So, because of the infinite, low cost distribution, the superstar or winning product can have an enormous market share, limited only by other competing superstar products.

Interestingly enough this economic inequality is tolerated better in the digital world than the real one, because everyone feels that they could create the next superstar product.

The addressable individual

Traditional marketing methods are dying out because of the many fragmented channels of social media and so content marketing is the new marketing and it is big business. The global sponsorship sales director at Manchester United describes fans on social media as addressable individuals with whom the club can have more intimate relationships [because with every interaction] we build up knowledge about who that fan is and what type of content they like to consume. With tailored content, the club encourages its 659 million worldwide fans to feel that special Man U connection by buying branded products endorsed by the club, and then share this feeling with the fans around them. If Man U gives something its stamp of approval then that creates value.

A common thread running through all definitions of social media is a blending of technology and social interaction for the co-creation of value. Rather like the shift in art culture, and everyone choosing what to treasure, we now have everyone creating content and sharing it across social relationships. But, even in this equal world, we have those more equal than others, with their influence and their ability to create content and also to make money – again they are tolerated more readily, because anyone has the chance to become the next great influencer or superstar.

Prosumers at playbor and weisure

As the role of consumer and end user disappears, the distinctions between producers and users of content fade. In many spaces online such as Wikipedia, users are also producers of the shared knowledge base, regardless of whether they are aware of their role. They are produsers or prosumers who collapse the gap between producers and consumers.

Often prosumers don’t have well-defined jobs in the 20th century sense of the word. The Internet allows them to blur that boundary between play and labour or work and leisure so they are in the environment of playbor or weisure.

@Stampylongnose, is a man in his twenties, whose job is to play video games in his bedroom. He creates a video of himself playing Minecraft everyday and uploads it to YouTube, and he has great fun and earns lots of money. On YouTube, he is more popular than Justin Bieber and seems like a really nice man. My girls watch him a lot, probably more than they watch television. But to them, because they often watch YouTube on our television, or television on a tablet or phone, it is all one and the same. They are digital natives, and for them, there is no distinction between Stampy, a superstar on YouTube and Katy Perry, a superstar popstar who puts her song videos on YouTube.

Convergence or Splinternet

My kids see the world quite differently from me. So, it is no surprise that sometimes when I talk about my childhood, one in which the Internet (well WWW, didn’t exist), they ask me questions which seem mad to me, but perfectly normal in the context within which my girls live: Did you have music, mummy, when you were little? The existence of the Internet is as normal as the existence of music. Using your phone to watch TV, is normal too.

Designer and web standards advocate, @Zeldman, says that there are 18,796 distinct Android devices on the market, and this will only continue to increase as technology gets cheaper and more widely available. So, although you can do similar things on more devices, known as convergence, more and different devices mean that everyone is beginning to experience the Internet differently. This different experience worries some, who wonder if such diversity contributes to the Splinternet and the fear of the balkanisation of the Web.

However, until now, no country has built an intranet, disconnected from the rest of the world. Many countries have blocked websites like Netflix due to intellectual property regulations and social media during times of crisis, which goes against all the ideas of what the Internet stands for which is no central governing body, and universal access for everyone.

One variation of splintering is the filter bubbles people can live in. Rather than the long tail of choice and diversity, Google, since changing its search strategies, now serves up more of what you have already seen rather than more of what is out there and Facebook has personalised newsfeeds. So, people can become less exposed to viewpoints which are different from their own and live in their own personal bubble, reading only opinions with which they agree.

Digital culture: Utopia or Dystopia

Social Scientists like Sherry Turkle, once believed that the Internet could help her learn about herself. Others believed that it would radically change our culture. In some ways it has – we have the weightless economy, and more opportunities to create jobs in a weisure space. We can publish our works and our art and we don’t have to wait for anyone’s permission. But for many, their lives are still mediated by the TV and newspapers, who tell us where to put our focus, even online.

And, the main downside of equal access for everyone without a central governing body, is that as the Internet has been adopted by the majority of the population in advanced economies, all of the inclinations, prejudices, and habits of society came online too. 

So in that virtual space where feminists can meet together and use social media to change sexist attitudes, we also have young women publishing erotic/sex confessional memoirs, pictures, videos and self-harm vlogs. Is this a disturbing trend or representative of young women today? In her book, Postfeminist Digital Cultures, Sociologist Amy Shields Dobson discusses young women and their behaviour on the Internet.

Similarly, with the Internet, we have a space free from social constructs which allow us to create different social structures. But in that space, there also exists groups of people who like patriarchy and hierarchy. We can have great discussions with like-minded people and find our tribe, but we also have trolling and bad behaviour far worse than what would happen in face-to-face discussion.

Without the Internet, you might never know that your lovely sweet old-lady neighbour is capable of saying the most awful things, in her second identity as a hateful tweeting Internet troll.  She is a great example of the post-structuralism (brilliantly explained in the link using hipsters) – a theory of the individual as an unstable entity in this digital world within which we now live. But without the Internet you might not have met those inspirational people either.

Individuals change, and digital culture, like culture itself, is constantly growing and changing, whilst everyone is renegotiating the rules of how it works. The Internet reflects all of this, which to quote the Eagles’s Hotel California: … could be heaven or this could be hell.

Thankfully, there is enough digital space to make it what we want.

Posted on

Simplexity and the Internet of Things

The internet of things pic

What is now proved was once only imagined
– William Blake

In the brilliant (alas, cancelled) Forever series, Episode 17: Social Engineering, Detective Jo Martinez, and her ME Dr Henry Morgan are called to the apartment of a young murder victim whose flat switches itself on whilst they are there. The morning radio comes on, the coffee maker starts to percolate, the heating switches on, and the blinds open, all controlled by an alarm on the victim’s phone.

The victim turns out to be a Faceless hactivist who hacks into Times Square’s billboards to play footage of politicians behaving badly, and into New York City’s municipal systems to alter everything from traffic lights to residents database information. So, it makes sense he would wire up his creature comforts to make his flat more ambient. The only downside, according to the NYPD cybercrime unit, is that his network was hacked. Someone logged in to turn on his boiler and cut the pilot light, which resulted in his being gassed. Murder by remote control.

The story might be fiction, but in reality, having a wired flat is very much a reality. According to the US Federal Trade Commission there are around 25 billion devices connected to the Internet which will double by 2050.

These devices are available in every context from heart monitoring implants to field fire-fighting search and rescue operations. Each device collects data and then autonomously flows the data between other devices using APIs, data formats and network protocol stacks in order to improve overall performance.

It sounds complex, but when coupled with a familiar device like a coffee machine and resulting nice ambiance, Mmm smell the coffee, the result is one of simplexity – an emerging theory which balances the need for simplicity and complexity, and the design focus for the future of the Internet of Things (IoT).

In June this year, Wired magazine produced a supplement about the connected home, 30-odd pages full of futuristic devices that are already on the market and connect to the IoT. A few of my favourites were:

  • The Triby Fridge Memo, an e-ink display you put on the fridge and when you write on it, it sends messages to the rest of the family.
  • The smarter am app which will customise your coffee, so if your fitness tracker says you slept badly, it will make you a double espresso to get you up and at ’em.

Gimmicks aside (cocktail mixer and fizzy water dispenser, yes please), a really useful one is the CO2 detector which in the event of an emergency would talk to you, your thermostat, and turn off your boiler.

The biggest problem us consumers have is deciding who will look after our smart homes. Is it Google with Nest? Or Apple and its golden handcuffs of proprietory software? Shame really, as these simplex gadgets have been around for many years just waiting on an industry standard to allow them to talk to each other.

But, it is not just the devices in this ambient intelligence interestingly enough which need monitoring, it is us humans. HCI designers have been saying for years the human is a factor in the design. With IoT, this is truer than ever before. Humans become devices to be monitored.

One way is with physiological computing. The physiology of a human is monitored and used as input to a system. So, if you arrive home and are a bit hot, your home might turn the heating down. Or, a computer game could modify its level of difficulty according to the amount of times you shake the controller in frustration.

Feeling wired: The human as a thing

Recently, Douglas Coupland asked in the FT: How much data am I generating? Involuntarily and otherwise. Everywhere we go, we generate data with Oyster cards and shopping bills at Tesco. Coupland wonders which algorithm is at work mining away in some big data pool in order to learn everything about us. His main fear is that it will all be monetised and we will end up being part of some sort of pay-per-click click junkies.

Ironically when I reread Coupland’s article online, it kept asking me if I wanted to tweet a quote. And, often when signing up for something online, I am asked to share this with friends on Facebook. Just imagine a wired house of consumer products: You’ve just left a note on the  Triby Fridge Memo, share this with your friends. Your coffee is a double espresso today, tweet this to your boss. Gah!

But, it is not just posting online which causes oversharing and potential security risks. Many people don’t change the settings on their new devices when they bring them home. So, devices are left to broadcast openly across the Internet which allows a would be burglar to scan local IP space and then gain access to footage of people at home, build up a pattern of behaviour and then break in, when everyone is out. To say nothing of the virtual visitors who tiptoe around and tamper with your systems when you are at home.

But even those humans who change the passwords on their devices, might still write their passwords on post-it notes and stick them somewhere everyone can see them, or worse still, use the same password everywhere. Designers know that humans are the weakest link in any system, which is why biometrics are being proposed as the way forward. If we use what humans have already it will be less painful than implanting chips under our skin or needing to remember our wearables.

We are all unique

We are all unique, well, not really, our fingerprints, contrary to popular belief, are not unique identifiers, but the retinal scan has an error rate of 1 in 10 million – not bad! Even so, if someone wants to access your system they will. Using brute force attacks as a starting point, it is easy to imagine someone compiling a database of fingerprints or even retinal scans to virtually or physically enter your home. To counter this, unique biometric identifiers are being explored such as gait analysis and Nymi’s heartbeat recognition.

Say the intruders have got in and left with your best kit, everything is not lost, the broadening application of block chain authenticity could help you retrieve them. It is possible to stamp your devices, rather like your bicycle.

Up until now Bitcoin has been used as cryptocurrency a form of money that can be transferred securely and anonymously across a widely distributed peer-to-peer network. The Bitcoin blockchain is an auditable ledger of all the transactions that have occurred on the network so far. It is a trustless system because the Bitcoin network itself is guaranteed to keep a fair and accurate record of which bitcoins belong to whom. Removing the emphasis on currency and keeping the blockchain technology, it is possible to track the history of individual devices and keep a ledger of data exchanges between it and other devices, web services, and human users.

The only downside is creating massive data trails, but when you have lots of devices in your home, your office, and in cities talking to each other, to humans, to the Internet, well we are talking a lot of data anyway. Plus more machines needed to process it into something meaningful. How much energy will the IoT need? Currently, 25% of UK energy is consumed in the home and this will only increase.

Sustainability in simplexity

Panasonic in Japan has created the first sustainable smart town called Fujisawa. It is built on the site of an old Panasonic factory and is designed for a population of 3,000 people.

The town has a smart grid with everything connected to it. Each house and apartment block has solar panels, and fuel-cell generators which generate and redistribute energy around the house, and then the town grid juggles all these variables of renewable technology and town demands.

Engineers anticipate a 70% drop in each house’s carbon footprint, and have anticipated earthquakes too. Enough power can be stored for three days of off-grid operations.

And, this is where the IoT gets a whole lot more interesting. If we can use technology to generate energy and redistribute the resources that we have across towns and eventually countries, then there is the hope, that one day everyone the world over will be able to wake up in a secure home to listen to the sounds of their creature comforts making their home an ambient one.

The IoT has the potential to redistribute the future more evenly. Simplexity at its best.