Fredrik Stenbeck

A Great Simian or just a Monkey

corporate collaboration

Corporate Collaboration is still broken

After all these years and new tools constantly being released, corporate collaboration is still not solved. I would actually go as far to state that almost nothing has happened in the last 5 years. Collaboration in large corporations is unstructured and scattered all over (both tools and information), without any form of measurement of productivity, information or impact.

Let us start with why we are using corporate collaboration tools within our organisation.

  1. Make better and faster decisions based on the right information from experts in the area.
  2. Become more productive.
  3. Tear down corporate structure and connect the right expertise / people and information without corporate structures in the way.
  4. As well as soft things as a sense of belonging and be part of a team etc.

…but at the end of the day, for a corporation, it boils down to being a more efficient organisation that research, produce, market, sell and support it’s products in a more efficient way so that the company performs better in terms of their corporate objectives.

Is there a corporate collaboration tool today, that can prove that they can measure the impact and / or result of their collaboration product if used in the right way within a corporation? NO!

Why is Corporate Collaboration not evolving?

This one is naturally a hard task to describe and I do not have a definitive opinion, but with a brief look at how things have evolved in the last few years, we might have some guidance.

First, most of the tools corporations use are sneaking their way in through the backdoor by small teams that are tired of Microsoft Sharepoint document-centric collaboration. These teams simply want a smoother and more efficient way of communicating and getting things done. Tools like Slack and Trello are sneaking their way in. A manager for a small team is adding the monthly fee on his credit card as an expense report and suddenly these tools have made their way in through the corporate door without passing the IT department and other slow and backward-looking departments. These tools are great in many ways and provide clear value to the team, but they do not provide any measurable value from the company perspective.

Secondly, these tools are built to assist small teams or multiple small teams. Multiple small teams do not mean that these teams are connected in any way. The company that suddenly have hundreds of small teams in e.g. Slack, do not gain any immediate measurable benefit from all these teams and channels that are full of one sentence communication and integrations with other tools. Furthermore, it is a tough thing to re-use the information in those team as best practices or similar for others to learn from.

What we have now is multiple consumer-centric social apps that have entered the company without providing any measurable benefit to the company (they do provide value, don’t get me wrong). On the other end we have lots of old-fashioned corporate software vendors trying their best to keep up, these softwares are either too boring and complex for people to actually use (or implement) or they are too simple and just trying to replicate these new shiny toys like Slack or Trello et al and by that also falling into the same category of non-measurable tools available. Microsoft Teams (Yammer) is one of those. Teams create the same value as Slack, but it is still not built to support a large corporation, it is for teams without connection to other teams or aggregation for corporate review to learn from, track or simply to give praise when a team is outperforming or support when they have a challenge.

Is there a future for Corporate Collaboration?

Absolutely. Even though the first paragraphs in this post are a bit pessimistic, I am fully convinced we will see lots of innovation in this space. AI (let us stick with this very general term for now) is one enabler and when the term social finally seems to be a natural part of the corporate infrastructure in one way or the other, we can start to build real value, from a corporate perspective, from these tools or new ones.

The ideal corporate collaboration tool

A dangerous headline on a paragraph, but conceptually we are now mature enough to take the next step and implement a corporate collaboration tool that is measurable and also prove the value of using the tool for the company as well as for the individual, the team and the department etc. This without compromising simplicity and ease-of-use.

Measurable

A modern tool needs to be able to measure it’s the benefits of it’s usage in real-time and present it to everyone.

  1. Individuals
    Are your contributions in discussions helping improve the task at hand and how do you measure towards your goals. How are your conversations evolving over time? Have you considered a person making others smarter or making others smile etc.
  2. Teams
    Is your team solving tasks according to timeline and dependencies with our teams / projects? Is there people in your team that provide value in these tools that do not get the right appreciation from their manager. How is the sentiment and tone in the teams and how do topics discussed map towards company objectives. How does your team rate towards other teams etc etc. Does your team provide value to other teams.
  3. Management
    Dashboards for everything with a real-time view of the operational status of the company. Find which teams, individuals, departments that are most productive. See how topic, activity, tone, sentiment etc map towards productivity. A clear view of great performing teams and create best practices etc. Find individuals that usually not get credit for their work, since they work in the quite, but often provide the information that makes the greatest impact on projects or similar (as new input to the yearly review with the boss).

Suddenly we have KPIs and real-time dashboards that can measure productivity and collaboration in a way that is not done yet.

Simply put, it is all about connecting the dots, that can partially be letting AI work the unstructured text, but also programmatically connecting the dots (either by user assistance or by code), a challenge YES, impossible NO. Still with the same usability as Slack or Trello et al.

Are we going to see a product like this soon? I certainly hope so, I know many organisation would love it.

Or maybe it is just me that thinks corporate collaboration is broken and a tool like above would provide great value to every corporation?

Deleting Facebook

I am leaving Facebook and Twitter

When presidents can publish threats and aggressive tweets on Twitter and not be closed down, I do not want to be a part of it.

When most information I see is angry post claiming all things are bad, I do not want to be a part of it.

When all discussions are polarized and you are expected to choose a side, either with or against. I do not want to be a part of it.

When you cannot say anything without someone feeling aggrieved. I do not want to be a part of it.

When information is algorithmically filtered so we mainly get information that supports our already existing opinions, I do not want to be a part of it.

Maybe the thing that makes my decision easy is that these services have a revenue-model that thrives on every single one of above topics, I do not want to be a part of it.

We have forgot that we are the product. Our information is sold to whoever pays.

It seems these platforms have become too big for us to remember that we are the product. These companies are selling you and me. We are the product and it is the data that you and I provide that they sell, that is why these services are still free.

The new normal is that we are on Facebook, if you are not there, you are odd. So, when they do not meet up to the responsibility that is required with 2B users, I do not want to be a part of it.

In my view, both these companies had a great opportunity to do good in the world (and have done so many times), but when they got listed and shareholder value became #1 priority, they lost direction. They thrive on Trump, extremists (in all directions and opinions), polarization, filter-bubbles etc. Not only does the engagement we see today have a really aggressive tone, but the worst thing is that these companies profit from it.

Regardless of above, I feel that these platforms have also lost me as part of the engagement. The ones that discuss are the loudest ones when it used to be a place for many that were not usually heard. I want these platforms to enlighten me, I want to be happy, I want to learn. All these things are lost on me on these platforms today.

I am today leaving Facebook and Twitter. Is it forever? I do not know. I know I am loosing out on some quality discussions amongst friends as well, but for me, it is not worth it anymore. I am out.

What I miss from my time in the Military

For a long time, I have sincerely felt I made the wrong decision when I left the military in 1998. I miss the passion, friendship, adventure and also how there was not an option to miss on solving the task at hand and never let things slip, not to forget the trust amongst colleagues and the constant flow of constructive feedback that was both given and taken. I know it is many years ago and a lot of things have happened since, but I really miss that job.

Why did I leave then?

There is no simple answer. When I left 1998, I had worn the uniform almost every day for 6 years (transferred from full-time officer to officer in the reserve, until 2007 when I resigned fully). At that time I felt I had to make a personal decision on how I wanted my future to look. In retrospect, I made the wrong one. Why did that happen? Well, I was young and restless…and to be honest a bit too restless to make the most sensible decision.

Simply put, I should have stuck in there and had the patience to really decide on what I wanted. I didn’t! My bad!

What I miss?

Me and one of my colleagues and friend (still in contact). From an issue of Arménytt about special forces probably in 1994.

This one is easy. I miss the comradeship, the passion, the adventure and the mental and physical challenge. I also sincerely felt I was really good at what I did. I was good at running our missions (tasks our unit worked with was terrorism, sabotage, subversion (similar to propaganda, spreading rumors, fake news etc), intelligence and crimes).

I was not a perfect military officer in any way, I had many flaws, which I guess both old colleagues and soldier can confirm. I had no ambition of becoming a general or high ranking officer, I wanted to be in the field, I wanted to be with the troops, I wanted to be in the frontline solving the tasks with the units, not behind a desk. I was probably considered by many as a soldier first and an officer second. It was certainly a lifestyle and even though I did not have much time off-duty (often by choice) I ended up at the regiment most days I was off duty as well. Simply because I wanted to….and my friends were already there.

A civilian workplace is a strange place for a former officer

In the military, I felt that I shared my passion and expertise with every single officer I worked with in those units. This is one of the things that is so flawed in the civilian workplace. Colleagues get away with being lazy and not doing their job. How the hell can that happen? ..or be allowed, for that matter. It simply seems people go to work to get a salary, not due to passion (at least a little) or that they like what they do.

Just look around yourself today, I am 100% sure you have a few colleagues that you sincerely think are lazy fucks and / or simply bad at what they do. The scary thing is that it is allowed.

Yes, we had different levels of ambition between my colleagues in the military as well, but I can honestly say that none of them let things slip, lacked ethics or lacked passion. A work like that is almost impossible to do without passion and ethics.

This created a huge respect amongst colleagues which I have not seen since. I miss that dearly.

I have never felt the same trust and comfort in my colleagues as I did during those years.

Don’t misunderstand this, I have had great colleagues and partners over the years, but it is hard to replicate that trust you build amongst colleagues in the military.

I also miss constructive feedback. This was a big challenge for me as a manager when I left the military, giving constructive feedback to colleagues and employees, it created a few strange situations, to say the least. Even given it is almost 20 years ago since I left, I still see this as something that the private sector can learn from the military. If no feedback (or bad feedback), it just creates backstabbing, confusion, rumors and bad work environment. I have seen it quite a few times.

What I did wrong after I left

Me and Anna probably 1997.

I did not keep in contact with my former colleagues, even though many of them really tried to keep in contact with me. That I regret deeply. At that time I decided I had finished that era and needed to move on. Many of my friends that also left, returned to the military after short stints in the private sector and many, that now have left, today work in sectors that are very close to the military (private security companies, chief security officers at large corporations etc). Many have naturally left for other sectors as well.

I am very confident that a military officer is a special individual, by that I mean, he / she is a tricky beast, not a simple individual. If you chose that line of work, you are different, simple as that. This is especially feasible in units like the one I belonged to and similar units.

This “different” thing also makes you an outsider at most workplaces, it often feels like you do not fit in. You get frustrated when co-workers don’t deliver on time or simply ignore tasks assigned from meeting etc. You can’t ever get your head around why we do things without a goal and why it is OK to simply let things slip between the fingers.

Most of the time my experience from the military have been highly appreciated by the companies I have worked with and the co-workers I have had, and I am grateful for all the knowledge and experience I gained. I think the experience a military officer has is an extremely valuable asset for any company, not always comfortable for the company, but very valuable.

What I brought with me from that time

I have still worked best in the frontline with the units (in this case co-workers or co-founders), utilizing my personal skills, but now in business development, sales, management etc. Similar in many ways, but also very different. I have enjoyed building my own companies / startups, and always worked towards large corporations as clients. I have always been the one building the business and closing the contracts.

Still in the field with the soldiers solving difficult tasks.

Realistically then, what should I have done differently?

The main thing I should have done was to keep in contact with my colleagues from that time. I still have contact with a few, but most are in the wind. It would mainly have kept the friendship going, but it would most certainly also have opened up for opportunities to work with these former colleagues. This in private companies as co-workers, starting companies together or simply work together in whatever business. Instead, I choose to venture off in tech and entrepreneurship without much contact with my previous life. I am 100% sure I excluded opportunities that would have given me the stimulation similar to what we had in the military if I would have stayed in closer contact with my former colleagues. Again, my bad!

I think I just needed to get this out of the system after all these years. And it is never to late to re-connect!

Top image: The medal you get when leaving Försvarsmakten. The one in the image is mine.

cognitive enrichment of unstructured information

Why is unstructured data so important?

Your business is making decisions only on 20% of the information you have access to, this since 80% of your information is unstructured and up until now not able to be fully utilized. It is about time we start to make decisions for our company based on all information we have, not only 20%. All else would be quite stupid, wouldn’t it?

Companies have tried to make sense of unstructured data for ages, but 78% state that they have little or no insight into their unstructured data.

It is an understatement to say that most of the worlds information is completely un-utilized and hidden in the dark.

You might think “I know my data” or “We can search our documents”, but that is not the same as getting value from the information.

What is unstructured data?

It might be clear to many, but just so we are all on the same page, unstructured data is images, video, sound and documents like blogs, news articles and Word documents et al.

An image is most often stored only with some metadata attached to it. That data is only telling us what time, date the photo was taken, sometimes (if the camera has the feature) it stored where it was taken etc. What is it in the photo? The most important information is completely hidden from us and we need to manually look at the picture to decide what it contains….it is the same with video.

For text, it is hard to find entities, sentiment, emotions, categories and also how they actually relate to each other. Which information is the most significant in a text and how does that relate to a target entity etc.

But I have Google Search?

Yes, we all do, but let us try an example. If we let Googles algorithm read all the Harry Potter books and at the same time let a cognitive system like Watson read the books, what will the difference be?

A simple yet powerful result is that one of them will be able to answer this question:

“Which house in the Harry Potter books is evil”

Image courtesy of Warner Brothers

It is not stated in the books that the evil house is Slytherin, but we all know it since we have read, reasoned and decided that Slytherin is all bad and Griffindor is good. That is an advanced example but still, puts the finger on the difference.

Google can deliver this result as well, but only if someone actually has written that Slytherin is evil in a text.

 

In a company context then?

If we translate this to a company, we could have thousands of reviews of our products stored in documents, but actually not know which one that is most appreciated and why (most reviews rely on stars, numbers etc to create a forked way of rating, but that does not tell us anything about context).

If those reviews would be enriched with cognitive information, the answer is only a search away.

Other examples are accident reports for insurance companies, customer support, legal (laws, regulations etc), social media, integrate unstructured data in business analytics and predictive analysis, medical research, product information, marketing and communication etc.

Examples – Getting value from your unstructured data

Example with getting value from unstructured text: A customer survey or customer feedback or similar. You receive a 2 star review and that is not good. What you are missing when not working with your unstructured data in a way that you can pull value from it, is that in the comments it says “The product was a broken unit, but Julia really went the extra mile to fix it” or a three star with the comment “Your opening hours make it impossible for me to contact you in any way, even though I love your product, I actually just bought 2 new ones”

What a traditional system misses is the following:

  1. Julia did a great job
  2. Reason for the 2 star was that the product was broken
  3. Opening hours are bad which pulls down the stars
  4. The average review was not connected to the product, which seemed to be a 5-star experience.

That is without a doubt important information for any company working with customer experience.

Example with getting value from unstructured data in images: Your ad agency has taken a bunch of photos for your new products and it is time to add those to the product information data. Often the data from photos and product data are disconnected, but no more. Now the process can be streamlined. If we put this in an online perspective it will not only make your process more efficient it will also increase conversation and increase sales, why?

  1. If a potential customer is looking for a yellow chair, he / she will find it immediately, this instead of browsing through pages of chairs of different colors and sizes.
  2. Since the unstructured data has become structured the Google results will increase significantly and your customers will find the yellow chair much faster.
  3. Value add and up-sale. Since we know the image contains a yellow chair, we now automatically can add value by showing products that fit well with the yellow chair and not only additional chairs as often is the case today.

Example with internal company documents: You have thousands of customer reviews, but they are only available in document format and poorly tagged, you only get product, date and some other basic meta data. You cannot get an overview of which products are having problems and with what, which products are highly appreciated and why, is it a specific issue that is re-occurring? If you enrich all your reviews with cognitive capabilities you will get the following (please note that this is not a huge effort):

  1. Dashboards with clear overview of all products and how they are perceived with a score.
  2. If problems with products, the actual problem is defined on the affected products.
  3. Image information of attached images can be analyzed. What products, color, model, issues etc can now be identified.
  4. …well, you get it.

How do I start to take control over my unstructured data?

All this must be complex, expensive and take ages to get up and running. Not really, the actual enrichment is very straightforward. For text, use the Watson Natural Language Understanding service. Send text through the API and enrich the document with the response from the API. You can also bulk upload documents (in many different formats incl .doc .pdf and HTML) to Watson Discovery Service if you want the service to manage the processing for you (ingesting, converting, enrich, store and also the querying). Watson Discovery Service uses NLU for enrichment, but also adds an end-to-end solution. The actual enrichment is the same. Using WDS is a bit more complex, but on the other hand, you will have your own cognitive search-engine in-a-box incl a powerful query-language) and intuitive tooling

If you want to enrich documents with domain-specific information like your own products, domain language etc it is possible to add custom ML-models to both Watson NLU and WDS through Watson Knowledge Studio, which is an easy-to-use interface to build a custom ML-model (done by subject matter experts, not programmers).

For images, it is a similar approach, but with the Watson Visual Recognition API and enrich the image with the response from the API. It is also possible to build your domain-specific classifiers so that Watson can recognize your products etc.

Conclusion

There are vast amounts of value hidden in unstructured information and in this post, I tried to take a few simple examples. In each organization, there will be easy wins, but as with everything, also more complex.

The best value is naturally gained when all information is integrated and put in context.

Today, only 20% of the information in companies are accessible, that will not create the most reliable foundation for a business to rely on, so it is time to get started to gain value from ALL your information.

natural language understanding in business

Business Benefits of Natural Language Understanding

Yesterday I wrote a post about Google Natural Language vs Watson Natural Language Understanding, Given that it was 2178 words you might think it contained a lot of info on why understanding natural language from unstructured text…….but no, so thought it was appropriate to write one.

I wanted to elaborate on some business examples in this post and below you will find two examples: Product Information Cognitive Enrichment and how to Utilize Natural Langauge in Customer Relations.

What is Natural Langauge in this context?

About 80% of all data in the world is unstructured (dark data), that is information that is in text (like documents, blogs, intranets, social media etc), video and sound. This information has been difficult to draw value from, up until now.

The concrete result of utilizing a natural language service like Watson NLU or Google NL is that you can surface information that previously was hidden and not accessible to draw valuable and actionable insight from.

Things you can enrich your information with is the following amongst other things:

  1. Is it a positive or negative text?
  2. What emotions are present in the text, is the writer angry and what is he / she angry about. Is it a product or a person etc.
  3. What products, persons, companies etc are mentioned in the post.
  4. What category does the text belong to. Is it a text about sports, business, tech etc.
  5. How do things relate to each other, is a person angry about a product from a company in a specific city?

But what about classic search, like intranet search or Google search? Can´t Google Search already do this?

Neither of these are good at managing these types of questions, they are good at relevancy, but not in understanding the meaning of the text.

Try imagining the result of this question in a Google search or intranet search:

“Show me the 10 most positive reviews of hotels in Stockholm that are close to the Royal Castle”.

If a service like Watson Natural Language Understanding have enriched all those reviews, an answer is returned in a wiff with ranking and everything.

Other examples can be:

  • “Which of our products are mentioned in negative terms the last week that is related to our new shop in Amsterdam”
  • “Which of our products are most mentioned in business-related articles “
  • “Which brands are most related to our product X”
  • “How does our product compare against our competitor X in terms of sentiment in the financial sector”

What about measuring mentions in Social Media etc?

It is also important to note that a simple question like “Which brands are most related to our product X” is not always a simple search for mentions of the two entities (brands and product name), but two things are different from the “dumb” keyword search.

  1. Is the product and the brand actually related to each other in the text?
  2. Is the product or brand really a product name or brand name? Let us take two Swedish companies as examples, Ericsson and IKEA and use the following two sentences as examples:

    “The politician Peter Ericsson and Ivar Andersson are traveling to Kivik on Tuesday for meetings”.

    “Ericsson has signed a new agreement with IKEA to implement their technology in the Kivik product range starting 2019”.

    You can replace Ericsson with Apple or many other companies, same frustration. The IKEA product can be replaced with most companies products since many have names that already exist.

What a natural language service does is that it understand that the first example with Peter Ericsson is not the company Ericsson, but a person. It also understands that Kivik is a city and not a product. Therefore it does not show up in a cognitive query, but would most certainly show up in a classic social media mention search.

In the other sentance, it understands that Ericsson is a company and Kivik is a product, that is Natural Language understanding and it will have a great impact on the value we get from information.

Integrate cognitive enriched information in your business applications

Now lets put this in a wider perspective where this information is combined with other sources of information like already existing solutions like structured data (BI / Analytics etc). Most companies can predict their product sales, but uncertain things always appear and most of the time it is hard to know why, at least prove the gut feeling you have, with combining enriched unstructured data with existing structured data new insights surface.

Example: Suddenly and unexpectedly sales drop for a product, this without any logical explanation. Predictive analytics can only use existing structured data to present predictions from, the unexpected is hard to predict and to find an answer to. This no more. By using customer service conversations and external sources like weather, news, social media etc we can suddenly see that the product is mentioned in negative terms and that it is due to a new feature that was launched a few months ago, but due to the change in weather seasons, the reaction did not surface until now. This information was found in customer service conversations as well as in social media information and weather data and when combined a the root-cause was found and the product could be fixed and sales returned.

It might sound like a far-fetched scenario, but did not want to make it too simple since most companies are complex and want to draw insight from the reality not only a simple “What products are mentioned in negative terms on Twitter”, but rather put it in context and connected with sales and the “why” question as well. The example above surfaces actionable insight that was not possible before.

Product Information Cognitive Enrichment

Product information tends to be very static and does not always match how the customers refer to the product. Often product data has internal terms and a lot of unstructured data is not possible to find.

Let us say you are looking for a yellow chair from a furniture store. We are heading to Google and enter a search for a yellow chair for bedrooms. I have tried this search with a few furniture companies and it is very similar, the result is that high in the ranking you first get the chair category and then the bedroom category (or the other way around). When you click a link and move on to the site it is often not the yellow chair you find, but a category with 30 pages of chairs or similar, somewhere we have lost the connection to the bedroom as well as the colour yellow.

I know all companies product data is different as well as how they work with Google, but if you work at a large company that sells products, I think you can relate to the challenge.

What I discovered when I tried several alternatives is that Pinterest surface at the top for these types of searches on Google, why?

Simple, it is because Pinterest has user-created product information. To put this in context for this post, the information on Pinterest is by default already cognitive enriched information, the product description and data is from humans, not a product database with only structured data.

So, if our product information would have been enriched with cognitive capabilities, how would that look.

Product data that is enriched with cognitive capabilities would connect the customer with the right product in one click instead of endless scrolling and headache.

Since this enriched product info know that the chair in the image is yelly and also know which chairs that is well suited for bedrooms, it is simple query to answer.

The result is higher conversation, more sales and happier customers.

Nice and dandy, absolutely, but is it really that simple? Overall I would say yes! If we break it down there are a few actions that is needed.

  1. Enrich existing product information with cognitive information from unstructured data such as editorial content, product images and existing product information. Could also be combined with behavioral data from Google Analytics or similar.
  2. To gain full value subject matter experts (in this case probably interior designers and product salespeople or similar) need to train a Machine Learning algorithm that understands what furniture that fits with others and also what colors that match other colors. We need to teach the algorithm to act as a designer. This to replicate the inspirational feeling we get when moving into a furniture store and not to put a pink toilet brush by the bedside.
  3. Access to data. Sounds easy, but can be a challenge.

Natural Language in Customer Relations

To enrich all customer interactions we have is also a great example where natural language creates value. Which customer service ticket should I start with, is the customer angry or happy (or angry when he / she started and happy when the ticket is closed), is it a critical business problem, is it regarding a high priority product etc etc. Is the answer already existing in our knowledge base so we can speed up the amount of time utilized on the ticket?

And equally important, all the information that was previously hidden is now available for us to use in dashboards for insights, customer satisfaction (without surveys!), product problems, time saved etc.

To enrich unstructured information with cognitive capabilities is of value for most companies and I dare to state that every single company can benefit from investigating how it can benefit your organisation.

So, lets start to empower your organisation with the real value in your unstrucutred information.

 

Top image from my favorite coffeeshop, Koppi in Helsingborg

google nl vs watson nlu

Google Natural Language vs Watson Natural Language Understanding

The competition in understanding natural language from unstructured text is thickening. Google just launched two new features for their Google Natural Language API, categories and sentiment. Those have been in the Watson Natural Language Understanding API for a while now, but let us see how the two APIs compare to each other overall.

Let us start with a head to head comparison with a real example.

Google Natural Language API vs Watson Natural Language Understanding Head-to-Head

I thought an article about another player in the game could be in order, so I entered an article in Fast Company about the Microsoft CEO Satya Nadella “Satya Nadella Rewrites Microsoft’s Code”

I will use the demo-interfaces for both services, they can be found here for Google NL and here for Watson NLU. The only difference I could find in how you post information to the two services is that you in the case of Watson, just can post a URL to the API and Watson does the rest. It is a simple feature that makes analyzing of web pages much easier, but the result is the same and also, someone has probably already built something similar for Google NL and put it on Github. If you do try the services I suggest looking at the actual API results as well, not only the demo-interfaces since those only show parts of the results.

So, how did they compare?

Document Sentiment: 

Watson NLU returns a 0.19 positive sentiment on document level
Google NL returns a 0 neutral sentiment

…so very similar, which I would have considered very strange otherwise given the length and depth on the topic in the article.

Winner: Shared victory

Sentiment breakdown

Both services provide a breakdown on sentiment so it is possible to determine sentiment on entities etc, but Google NL also provides sentiment on sentences, which can come in handy since it puts the sentiment in context immediately in the result from the API.

Entities

I started by listing a few entities to compare, but it does not give a great perspective of the capabilities since those numbers need to be in context, do run it yourself and check the result for details. Overall they are very similar, naturally with some differences in the result, but overall similar. Watson NLU provides a slightly better granularity, but Google NL has the sentence result which is very good, so overall very similar.

Winner: Shared victory

Categories

Again, very similar. The major difference is that Google added the news and business categories, while Watson was a bit more rigid and stuck to tech and software. Even though the entities in the article mainly are tech-related I did like that Google NL classified the article as Business / Industrial at a 0.89 score, while Watson NLU did not include any business-related category, but classified the major category as /technology and computing/software at a 0.67 score.

Winner: Google

Entities

This one was a bit peculiar. Entity identification is naturally a difficult thing, but was a bit surprised by the results from Google NL, while Watson NLU was quite solid. Let us just look at the top 4 from each

Watson NLU (the score is relevance score)

  1. Microsoft, Company, 0.87
  2. Satya Nadella, Person, 0.81
  3. CEO, JobTitle, 0.55
  4. Steve Ballmer, Person, 0.38

Google NL (the score is salience score)

  1. Satya Nadella, Person, 0.47
  2. Microsoft, Organisation, 0.42
  3. learner, Person, 0.02
  4. CEO, Person, 0.01

The two things that surprised me was the drop in salience score already after the second entity, it stayed at 0 for all the rest of entities, as well as the type for “learner” and CEO….and also that “learner” was classified in that way at all. If I look through the entire list in Google NL, I cant get my head around it completely.

It also seems like Watson NLU has a bit better capability in business related types and Google NL is a bit more focused on consumer types. Watson NLU clearly more structured.

Winner: Watson

Conclusions of the test

The main differences between the two is that Watson NLU supports more features, like emotions, as well as the opportunity to apply custom ML models to the Watson NLU. This gives Watson NLU the capability of learning entities and relations in your specific domain.

Google NL has the benefit of being straightforward and support all their features in all languages as well as having a bit more granularity in their score (salience and magnitude).

Is it actually working? I would say that both services are good at what they do, but I would give the win at this stage to Watson due to the more extensive features as well as the capability of adding custom models. This is from an enterprise perspective, if you are in the consumer space it might be worth to do a POC on both. I like how IBM has started to be more modern in their approach with Watson and I think the APIs are working very similar. They are open, well documented and easy to work with (please note that I am not a developer).

Also, it is worth noting that much of Watson NLU have been around for a few years now (through the IBM acquisition of AlchemyAPI 2015 ). Google has been in the game for many years as well, but not in the enterprise space with a packaged service for natural language. If Google continuous to focus on this space I think they will be a real threat to IBM if they do not keep their pace up (which I see is a risk given it is IBM).

I would say as of this date, Watson NLU is the winner in the test, but I think Google is working at a high pace to package it’s extreme knowledge in the space quickly and I expect a lot of progress at a high pace. So, even if Watson is a leader today, they might not be tomorrow. The difference seems to be in the packaging, not the domain expertise.

For a bit of breakdown on pricing, terminology etc, keep reading.

What is Natural Langauge in this context?

Simply put it is the capability to do text analysis through natural language processing. It gives us the possibility to extract the following:

  • Entities
    Extract people, companies, places, landmarks, organisations etc etc
  • Categories
    Automatic categorization of the text. Both Google NL and Watson NLU has an impressive list of categories. Google state total 700 and I have not counted Watsons, but seems to be about the same.
    List of categories for Google NL.
    List of categories for Watson NLU.
  • Sentiment
    Is a text positive or negative, but nowadays it does not stop there, it is also possible to break it down further to target the sentiment at specific entities or words (differs between Google and Watson, more on that later in the post).
  • Syntax / Semantic Roles
    Linguistic analytics of the text by splitting the text into parts and identify nouns, verbs as well as subject, action and object etc. The Google Cloud Natural Language Syntax feature seems to be a bit more extensive than Watson Natural Languages Semantic Roles.
  • Keywords, emotions, and concepts (Watson only)
    Emotions are …. emotions like joy, anger, sadness etc. A great feature for customer service or similar products.
    Keywords are words that are important in the text.
    Concepts are words that might or might not appear in the text but reflect a concept.

Terminology

The two services use similar terminology. Google uses Syntax where Watson uses Semantic Roles, otherwise very similar terminology.

In Watson NLU all results are returned with a confidence score. Google has added two additional things to consider, magnitude and salience. Personally I like the simplicity in only using the confidence score, but naturally, the two other values can provide additional value in some cases.

Confidence Score: Is a score between 0 to 1 and the closer to 1 it is, the more confident it is. Usually above 0.75 is considered confident, but that is naturally depending on the subject and domain, you do not want a car to only be 75% sure that it is ok to do something, but if a customer service representative is getting a ticket that is 75% confident to be a Lost Password ticket, that will do.

Sentiment Score: Is a score between -1 and +1. When close to 0 it is fairly neutral, the closer to 1 the more positive and when close to -1 it is pretty negative. Watson actually sent the positive/neutral/negative-label in the API, Google only the score. Google Natural Language also sends a Magnitude parameter. Magnitude is a score to complements the sentiment score by telling us how strong the sentiment is.

Salience: Shows how central an entity is in the entire provided text or document. It is a score between 0 to 1. This is a good feature to if you need to see how “heavy” an entity is in a text. Only available in Google Natural Language.

To see explanations of Google Natural Language terminology as well as examples of JSON results for each of above, do visit Google Natural Language Basics.

To see explanations of Watson Natural Language Understanding terminology as well as examples of JSON results for each of above, do visit the Watson Natural Langauge Understanding API reference documentation. There is also an API Explorer if you want to play with the API.

Custom ML-models?

If you are an enterprise this feature is usually very important, this so it is possible to extract domain-specific entities and relations. If you have build an ML-model it is very easy to deploy it to Watson Natural Language Understanding, but I could not find a way to do it with Google Natural Language. Since I am not entirely familiar with the Google APIs I might be mistaken here, so feel free to correct me and point me in the right direction.

It might also be as simple as that IBM comes from the enterprise angle and applying custom models in more of a pre-requisite for IBM than for Google that comes from the consumer space.

Supported Languages

In terms of AI / Cognitive / Machine Learning the language is always a tricky beast. I have written extensively about what languages Watson understands, and will in this context only compare Watson NLU vs Google NL. I would say they are on-par with each other on this topic. Watson supports Arabic and Russia, while Google NL is supporting Chinese (both traditional and simplified). As a Swede, I will give Watson the victory, since Watson NLU actually partially supports Swedish as well, but that is a very biassed Watson victory.

Additionally, the comparison here is a bit difficult. I interpret that Google NL supports the listed languages for all features in the API, which is very good. Watson NLU has more features but does not support all features in all languages, so dependent on your task one or the other might support it.

languages supported by watson natural language understanding

Supported Languages for Watson Natural Language Understanding

languages supported for google natural language

Supported Languages for Google Natural Language

What is the price for Google Natural Language

Monthly prices per 1000 text-records. One text-record can contain up to 1000 unicode characters. It might seem complicated, but if you have followed my posts of pricing prior, it is clear that they all are equally complicated. Full details available at the Google NL pricing site.

google nl pricing

What is the price for Watson Natural Language Understanding

Watson NLU is also charged on a per “block” per month price-model, they call it units and a unit is about 10.000 characters, so bigger units. IBM also charges for enrichment features. As an example: if you want a 18.000 character text analysed for entities and categories, it is 4 NLU Units (independent on how many categories or entities that are returned). Two units for the text and two units for the features. Looking for pricing for the rest of the Watson APIs, I have a post with a spreadsheet with the cost for all Watson APIs.

Watson NLU pricing

Given that the prices for Watson NLU are labeled in Swedish krona (since it is my live Bluemix account I have taken the screenshot from), I also attached a simplified model so it is easy to compare to USD as well.

Conclusion on pricing

This is a tough one since these models are hard to interpret before you have worked with them live and actually been invoiced, which I have not from Google, but from IBM Watson.

Nevertheless, I get the impression that you get more bang for the buck with Watson in this case. I sense that the free tire is more generous as well. But, this is a tough one for me to come to a clear conclusion, so it is more of a sense than a fact that I think Watson is more bang for the buck. The day I will receive an invoice from Google with NL on it I might update.

Disclaimer: I have been working with the Watson APIs for many years and know them pretty well, I am not as deep with Googles APIs. With that said I am open to others to complement my analysis and / or conclusions.

Top Image: The image is a wallpaper from the game Crysis 2.

human-tail chatbot

Short-tail, long-tail and human-tail chatbot

I am not that overwhelmed by the hype of chatbots as a buzzword for AI. I see chatbots as an interface. It might be considered an evolution in terms of UI / UX, but as an example of AI, I am not convinced. So, what is the use-case for a chatbot then, in terms of AI? This is how I see it.

I have written about my thoughts on why I think a chatbot is a stupid example of AI, so will not go into that much further.

I am dividing the chatbot use-case scenarios into three different stages:

  1. Short-tail
  2. Long-tail
  3. Human-tail

short tail long tail chatbot

This chart simplifies my description. As seen in the chart, a well-implemented chatbot can save wast amount of time and help people focus on the quality work instead of assisting on simple tasks that re-occur very frequently.

All of above can have a chatbot as an interface, but can also be integrated into other existing software, be a classic webpage or an app, it does not matter, but for me, this describes the use-case for a chatbot pretty clear.

What is a chatbot?

This is also a term that is up for interpretation, but for me, a chatbot is a software that can understand the human language, understand the meaning and intention of what is said, identify entities and then respond in a way we understand as well as with the appropriate language for the domain.

Short-tail of a chatbot

This is the most common use-case and use-case with the least AI in. Short-tail answers simple, repeatable tasks, that are common and easy to foresee. Examples are:

  • What are your opening hours?
  • Can I book a table for 2, tomorrow at 8pm?
  • Who plays Harvey Spector in Suits?

From a customer service perspective, short-tail are often replacements for FAQs (internal or external) or the most prominent features on your company site.
Examples:

  • What is the wifi password
  • How do I configure the printer at 5th floor?
  • Show me product X for women in red.

As you see from above examples, this is not that much of AI except that the bot needs to be able to understand the intent of your text and potentially identify a few entities (like color, names, hours, dishes, sizes, product names etc).

Most chatbots we see today are in this category, not all have the ability to understand the meaning and identify entities, but still, those more “stupid” bots also fall in the category.

Short-tail chatbots are essentially the replacement for site-search and forms on sites.

Long-tail of a chatbot

Now we are starting to touch AI (or augmented intelligence) and the chatbot might provide more value than just being a more productive interface. The reason for this is that the long-tail chatbots can answer questions that are not common and questions that might be buried deep in all the unstructured data (80% of all data in the world is unstructured) we have, usually impossible to find since up until now, our search-features have not been able to understand, reason and learn knowledge in specific domains, today that is possible, that is what we tend to call AI.

This is a chatbot that actually tells us things we do not already know.

A short-tail chatbot only makes a process a bit more effective and streamlined in a simple user interface. A long-tail chatbot actually provides real knowledge and makes it available on-the-glass for us.

A long-tail chatbot takes much longer to implement given that we have to train the bot on the domain that it is going to work in. This is done with subject-matter experts. Often a new ML-model is needed for the bot to be able to fully grasp the domain and be able to understand, learn and reason. The ML-model is often also needed for the bot to be able to understand the more detailed and niched questions that might be asked. This since we still need the bot to be able to understand the meaning and intention of what the user is asking. Since long-tail bots usually are applied in a narrow field and with depth in that narrow field.

Human-tail of a chatbot

Remember the last call you had with a call-center? As soon as a question you have is not solved quickly you tend to end up in two scenarios, either you get angry or you are transferred to the manager (or you are informed that this is above the operators pay-grade and they need to talk to the manager etc). Let´s put this in the bot scenario.

  1. You get angry!
    A bot can today sense emotions and notice that you are either using a bad language (which we tend to use more frequent with a bot compared to on the phone) or that you simply are starting to show some frustration and irritation.
  2. The question requires manager assistance
    At a certain stage the bot might be given a question that simply is above the authority of the bot, what shall the bot do?

In both above cases, it is hard to train a bot to act accordingly since emotions are very hard to communicate in a chat, and even harder if you are a bot.

This is where the human-tail comes in.Human-tail is simply when a bot senses that it can no longer manage the conversation with a positive outcome. It is time to hand it over to a human. Some tasks are simply better suited for humans (still).

Natural human-tail scenarios need to be implemented in the bot as well. This can be done by alerting a human to take over the discussion and when the issue is solved, hand it over back to the bot. The human can see the entire conversation as well as emotions and all the different products, agreements and other details that have been either collected or pull from internal sources. Another scenario can be that for certain topics you get the option to be transferred to a human instead of the bot, this by choice of the user, not automatically.

Personally, I think the human-tail is as equally important to build a great bot, from an end-user perspective.

Augmented Intelligence

I have written about augmented intelligence many times, but most AI and cognitive solutions are implemented to complement and elevate humans, not replace. Therefore I like Augmented Intelligence better than Artificial Intelligence that often insinuates that AI is replacing humans.

In the above three, this is very clear.

  • Short-tail
    The bot simply removes the easy to solve scenarios from our lives and lets us focus on the scenarios that require more cognition.
  • Long-tail
    We, as humans, cannot remember everything and cannot learn everything. In the long-tail scenario, the bot is helping us with the things we do not know or that we simply have forgotten about. elevating us as humans.
  • Human-tail
    The bot acts as a first-line support, a meat-wall (or a computer-wall) to put it bluntly. We only get calls / chats which specifically require human capabilities. We are still better suited to manage a situation where emotions play a large role or to calm an angry person down. We also tend to be better at getting an angry person to become happy again, by explaining etc, a bot can occasionally be a bit rigid when it comes to what is right and wrong.

Photo taken August 2017 by me, in Vägerödsdalar on Skaftö, Sweden. It shows a direction pole for Bohusleden near our summer house.

Am I rich?

A recent trip made me reflect on the life we live and what we are searching for or trying to achieve with our lives.

As a person, I see my life as a journey of change and constant learning. I see the term rich quite equivalent to being able to choose, having freedom and be happy. As a westerner, choice and freedom often come connected with a financial situation that actually admits being able to pay for those choices. In the best of worlds, we work with things we love that makes your world and others a little bit better whilst also giving you the financial opportunity to pay for the choices you make. Yes, pay for, since most of the choices we make in the free and democratic world come connected with a cost.

But what happens if you do not have freedom, not the opportunity to choose or the financial situation to do whatever you want, are you less happy then!

If you read the picture I paint of what I consider ‘a rich life’ I would consider a life without the ability to chose or the financial situation to make changes a not so very rich life. Naturally, this is if you read what I write as the devil reads the bible, but in essence, it is actually correct.

So, after spending time in a village where they have freedom, but not the fortune to chose and not the fortune to be able to pay for their extravaganza (as if that was a goal), what happens?

I remember reading Dalai Lamas book “Happiness” and in the foreword, the author wrote that there is no connection between money and happiness, just an initial joyful feeling when you realize you can do all the things you previously dreamed of, after that the happiness clings off.

With that in mind, it is not hard to realize that my friends are far happier than most of us reading this post. Why? What I realized with the way they lived was that they are happy “in the now”. They are happy for what they have. Most of them live with their entire family, grandmothers, grandchildren and most of the time a few aunts and cousins as well. They all help each other and always without any claim for something in exchange, they work their asses off, but they do it with a smile.

They live in simple houses, their kitchens and toilets are all outside, they sleep on the floor and the food is all grown nearby as is the chickens and the fish eaten. The mother of our friend only eats things from her own farm, this to save all the money earned from the rice fields to buy more land to grow more rice.

Did I miss something whilst there……naturally I did, I missed a comfortable bed, I missed some food etc. Did I miss something really substantial? Honestly not a thing. My family was there with me and how they lived their lives made me so relaxed and comfortable that I enjoyed every single second there, we already want to go back and spend more time with these wonderful friends of ours and be more enlightened by their way of life, learn, learn and live in the now. Happiness comes from actions nothing else. Actions can be many things, but actions that enlighten us and actions that bring happiness will never be a bad thing, purely a good thing, then how we actually put these actions into the world we currently live in is entirely up to each and every one of us. Our friends sincerely made me realize that my actions are too small and too materialistic driven, they need to be driven by the right triggers in life.

This is probably the most fluffy post I have ever written and I can imagine that many of you who read this, just shake your heads and wondering what I am babbling about. Not to worry, I am still the same guy, just that I see life in a bit different and happier way and that I am very certain that the last couple of days have changed how I look at all the actions made in the world, nothing that probably will be feasible in my daily activities, but in my heart and brain it will always be present.

I am forever grateful to our friends that invited us and showed us such hospitality and also enlightened us about some really important things in life.

 

20 years ago IBMs Deep Blue beat Kasparov in Chess

https://youtu.be/wdns2fVUEeE

The 11th of May 1997 IBMs supercomputer Deep Blue beat the current world champion, Garry Kasparov, in chess. IBM Research released a little re-cap of the event and what has happened over the years up until today and Watson, that was launched to fame in similar manner, by winning Jeopardy in 2011.

Deep Blue actually lost the first game in 1996 with 4-2 in favor of Garry Kasparov, but in the re-match in 1997, Deep Blue won.

watson price and language update

Watson Language and Price Update

Since publishing the post on what languages Watson supports and how much Watson actually cost, those posts have generated outstanding most visits on this blog. Since Watson is constantly updated I thought it was time to update those posts since the Watson language post is from Dec 2015 and the Watson price / cost post is from August 2016.

I will in this post just point out some major updates and differences, the complete tables of languages and prices etc are in the respective original post.

New, discontinued and merged Watson APIs

14 Watson APIs.png

Today there are 13 APIs available with a lot of merging happening. Well, there is actually 14 listed today, but Tradeoff Analytics is already discontinued, so 13 is the correct amount. Just recently a few APIs have either merged or been discontinued. Dialog is now only available through Conversation (no more XML horror), Alchemy is fully integrated and all the visual / image APIs are merged to one. I like this change even though it is kind of the opposite of what IBM told us a year ago when the stated there would be 50 APIs released. To be honest it is a lot easier to work with 14 then 50, so great to see this merge happening. This might naturally lead to the notion that you pay for more features per API than you actually need, but overall IBM has lowered the price, so that is not currently a risk. I only found one service where the things had changed in a negative way, and then it was only the free option for Language Translation, it has decreased from 1.000.000 free characters to 250.000.

Update: What does Watson cost?

Below are some notable changes to the Watson pricing.

Natural Language Understanding: Compared to Alchemy Language, the entry level has decreased from $0.007 to $0.003 per call, which is a significant decrease in price. Secondly, customized models have decreased from $3500 to $800 so also a price decrease. Otherwise very similar structure.

Conversation: Price decrease as well, from $0.0089 per call to $0.0025 per call.

Language Translation: Primarily a 75% decrease in free translations from 1.000.000 to 250.000. The only service that is updated in a negative way.

Visual Recognition: More than 50% reduction in price for Custom Classifier Training per image. This is great since that is a key feature in Visual Recognition that no one else is offering. IBM also removed the fee for storing the custom model.

Discovery News: Is the old Alchemy News. Fee model is integrated into the Discovery service instead as prior in the Alchemy Language service.

Discovery: A new search engine service, so updated the table with the pricing for this service.

No change to the rest.

Head over to the updated post “What does Watson cost? What is the price?” to see the updates.

Update: What languages does Watson support?

Unfortunately not so many updates as one would have hoped for during the 1.5 years that have passed since my initial post on the topic, still there are some changes. First the documentation is now a lot better and most services have a “supported language” section available, not all, but most. I assume the merging of some services has enforced some structuring of both the APIs as well as the documentation, which is very notable in the Natural Language Understanding documentation. Prior it was scattered all over and documented in so many places it was hard to keep track, now it is all displayed in a nice table (which is included in my post as well). Outside of that, there are just a few languages added to the APIs.

In the table, I have tried my best to provide accurate links as well, so it is easier to find updates on the languages and to read more if needed.

Now, head over to the updated post “What languages does Watson support” to see the updates.

Page 2 of 8

Powered by WordPress & Theme by Anders Norén