Nivedit Majumdar Nivedit Majumdar

Making Email Replies Smarter, Through Context

It’s truly amazing to see how far Context has come through the years. With the help of bigger spheres such as Artificial Intelligence and Deep Linking, we’ve seen Context being introduced to common platforms so that they can be accessible to the maximum target audience. Be it in the form of virtual assistants, or smart feature of applications, one thing is certain: Context is here to stay, and it is definitely going to grow more in the time to come.

Incidentally, all these trends happen due to key developments and decisions taken by big corps – Apple and Google in particular. Previously, we’ve spoken about Apple’s take on Context/Deep Linking, and we’ve also seen Google’s view on Context and Machine Learning at I/O 2015.

But in this article, I take a look at how Email is getting a contextual makeover – not just in terms of categorising email, but also generating replies according to context. I’ll also be talking briefly about deep neural networks, and how concepts such as these might become huge in the near future!


A little more than a year ago, Google took the internet by storm when they launched their new email centric application – Inbox. While it offered pretty much the same features as any other email client when it came to responding and syncing, it took things a step further through context, as we spoke about in this article. But still, here’s a brief recap.

Inbox brought a new aspect of organisation as far as email was concerned – it analysed the content in email and categorised the email into bundles accordingly. Relevant content was grouped together, so things like travel info, or promotional offers, could be accessed from a particular bundle easily.

It also brought highlights – which brought details in the form of highlights without actually having to open the email. All in all, context was aplenty in Inbox, and with the latest update, it has actually increased.


With the latest update to Inbox, users will get three predetermined replies for specific types of email. These are short, concise and perfect – specially for people who get tons of email and are looking to cut down the time spent in the inbox. Basically, Inbox analyses the text content within the email, and generates replies accordingly. Smart!

(Image Source: Google Research Blog)

According to the developers, the intent behind this particular feature was to make the whole ’email on a phone’ experience more simplified and seamless. Users could now reply to email faster without having to type out anything, and this does make sense if time is a constraint.


Consumers will find the three responses nifty, to say the least, but personally I find the whole mechanism behind this feature to be more appealing. And in this regard, the smart replies feature uses a multitude of concepts such as deep neural networks and sequence to sequence learning.

Here’s what happens: there is a pair of recurrent neural networks involved. These are basically networks where connections between units form a directed cycle. RNNs are truly intelligent; they exhibit some dynamic behavioural properties which allow them to process arbitrary sequences of inputs. And this proves to be beneficial in cases of unsegmented, diverse input sequences – such as handwriting recognition, or speech recognition. You can actually read more about RNNs here.

(Diagram by Chris Olah. Image Source: Google Research Blog)

RNNs – in the smart reply system – help to bring more dynamics into the algorithm. It is important to note that human interaction is being dealt with, and human interaction is never constant, it never follows a distinct set of rules. So it makes more sense to develop robust and dynamic algorithms accordingly. Ergo, RNN.

One RNN in the smart reply system analyses the incoming email – marking out relevant keywords and trying to analyse a pattern. This RNN is followed by a thought vector – which establishes logical connections between two similar – yet differently phrased – sentences. For example, the statements “Are you free tomorrow?” and “Does tomorrow work for you?” mean the same, but are phrased differently. If one stuck to a rigid algorithm to analyse these statements, it would be impossible to do so. But this ambiguity is removed through thought vectors.

And finally, the second RNN comes into play. By analysing the output of the thought vector, it synthesises a coherent and intelligible reply which would be suitable for the question that was the input for the first RNN. And according to the Google Research Blog, the detailed operation of each network is entirely learned, just by training the model to predict likely responses.

Smart Reply actually incorporates only machines and machine learning algorithms in the process, so no humans are ever involved in the interaction. Which means that no humans are reading your email. Besides, there was another issue of email being too long to fully decipher by the RNN, which is why the researchers at Google utilised a variant of Long-Short-Term-Memory, which preserved the long term dependencies – without being distracted by the less relevant sentences in the email.


Personally, I think technologies such as these present valuable solutions for the development of artificial intelligence and specially AI assistants in particular.

As was shown in the movie ‘Her’ , one might surely see virtual assistants gaining cognition and intelligence like never before, and while it does have the risk of gaining Skynet like nefariousness, it also has the potential of blossoming into intelligent features in applications and devices, which might help better human lives everywhere. And for this, I am truly excited.

Sign up for our monthly mailing list