Keynote: Knowledge Systems and AI

>>I am excited to
introduce our next speaker. Our theme has been how systems are fueling
future disruptions and yesterday we had Mark
Russinovich come in as the CTO of our Azure
Business to give you a view of how systems is
impacting our Cloud business. Today we have the CTO from our AI and research group
coming in David Ku. David is also the Corporate
Vice President for our AI Core Business, this is the group that powers the AI
capabilities behind Bing, Cortana, Office and Azure. It’s the group that designs, develops and deploys
the Bing Graph, the Bing ads marketplace, the office Knowledge
Graph and the substrate. David has always been
a big supporter of research as well and
our engagement with academia who’s the executive sponsor for our relationship with Stanford and he’s very engaged with a number of communities from
an entrepreneurship standpoint and connecting
high-tech communities into the Silicon Valley, and he’s an all-around good guy. So with that, let me
introduce David Ku.>>All right, thank you. I’m all around good guy. Well, good morning. I’m
very excited to be here. I would say in my many years
of working in products that I had really
the deep appreciation and pleasure to work with
researchers in academia. I think we’re all working in this very advanced field of
technology and products and that collaboration I just
deeply appreciate and the fact that Microsoft is so deeply
invested in research, in academia, I think it’s a blessing and
I’m certainly honored to be here and what I want to do is I want to focus
the theme on knowledge. Now this is from my experience
in working on in search, in advertising, as Sandy said in Office, productivity, Azure. There’s a theme which is around the ability to
semantically model and we’ll just call that broadly knowledge and I want to share
a little bit of the journey but also encourage and
hopefully frame that for AI transformations, both with the business impact and in terms of
experience impact. I personally think knowledge is one of the core
capabilities and it’s a rich area for which we’re still at the beginning
of understanding. So I enlist all your creativity to help us make
advances in this area. So let me start out by
just talking a little bit about the promise of AI. So there’s a number
that comes up which is 1.2 trillion, that’s, trillion with a T and that’s
the estimate that IDC has of the new incremental revenue that’s going to be created
in three years with AI. Now to do that and for this optimism to
really come forth, there’s really this belief that there is
unlocked potential in the data assets within an enterprise or within a company that they
can bring to bear, to gain new insights and to change the way they interact with customers and grow
their business. So that’s a massive
massive expectation. In the context of
our engagements with enterprises that are looking
to tap into the cloud, tap into AI in their desire
to transform themselves, we’ve seen a pattern and I want to list out
the pattern of what they’re looking for in their progression of applying AI to their business. Starts out with applications
that are intelligent. Being able to take
applications, point of sale, specific line of business applications
productivity and make that intelligent, feedback-driven,
predictive, analytical. But beyond this, there is also this desire to change,
use AI to change, the way they interact with
people and that could be employees or that could be customers and this is the way
of conversational AI, where it’s not just any set menu or interaction
metaphor it’s really just this language asking
interaction metaphor and that again we’re still
at the early stages of exploring that trend. Then there’s process
transformation like being able to understand not just
analytically retrospectively, the difference between BI and AI is in some sense
the ability to predict, anticipate and optimize
going forward, not just looking backwards. And then a general desire that the transformation of
a company is complete when every aspect of the system
of the company is somehow modeled that
allows us to reason, optimize and to drive. So, that way for
Business Value requires a company to really
deeply look at what it takes to be an
intelligent organization. So let’s dig into
it a little bit. It’s no doubt that
the first step to becoming an intelligent organization
or for a company to be effectively embracing AI
is to start with the data. Start with data and silo
it in many many companies, starting with product companies but also enterprise companies they need to start to now look at un tapping the value
of your data. So lots of work in understanding
the quality, the data, being able to connect
to all the pulses and the inputs both in terms
of real interaction with the customers and
internal processes and interactions and
to be able to have the ability to model and that requires us to really
have a foundation for rapid iterations experimentation
with all sorts of different modeling
techniques and so this modeling agility
with the ability on siloed data are table stakes. But when you start looking
at what it takes for a company to truly embrace AI, I would posit that there
is a third phase which is the ability to start
shifting the mindset, that data is in fact at the core, that there is a fly wheel
where everything evolves around not the existing interaction
with customers. When in fact there’s a knowledge, there’s a model of the business for which even
software and experiences are there to help us understand
that data and model better as opposed
to the other way around and that requires a really fundamental thinking of what is the core
asset of a company. Now, when you look at all
this and you say, “Hey, this seems fairly
abstract and how do we bring this to life is this really something that
happens in practice.” I want to share a little bit
of the journey that we went through at Microsoft and
certainly we’re not alone, so this is not saying
that this is unique, but I think it is indicative of this general understanding and the evolution of capabilities and mindset that is in fact changing the way
we look at the world. So let’s start with Bing. Bing is a search engine,
web search engine. You got lots of documents, you got queries come
in and in fact there’s a beautiful interaction
model of ranking feedback through clicks from
which you start building out a richer and richer
index of the web, of the documents and
the concepts within. So as part of that over the Internet as we did
several years back, probably ten years back, we started working on
knowledge graph because, hey! Why stop at just going to links, why not bring that information or start creating
that information or that action that
can be directly engaged without one hop away. So we start building
this knowledge graph. It turns out to be
a pretty big graph, two billion plus entities continues to evolve
across different domains, it’s open domain,
many many webpages, lots of techniques on it. But the thing that
is interesting is around a couple of years ago we started shifting it from, “Oh! That’s just a Bing graph,
that’s just a Bing graph. It’s only useful for Bing”
And we realize that what underlies it is in fact
a model of the digital world. These are the people, places and things that
happen on the web. These are visual artifacts
on celebrities, people, on locations, stores and when you look at
it from that standpoint, we’re starting to now clearly
understand both the facets the relationships of
things in the world, the public world and we
can start join it with structured data and as
a result that piece of asset, that piece of asset
created a life of its own. It’s now valuable
not just in Bing, but it’s valuable
whenever you need a model of the world which
could be an Office, which could be in Cloud and
that’s something we’re seeing this general pattern
that you start out with an application feedback, but you start creating
valuable assets and knowledge models that have life beyond that scenario and
let’s go into the next one. So, a little bit later my colleague Coshan
will come and talk a little bit about the
Microsoft Academic Graph and this one, think of it as, this is a graph that models research and technology
innovations and really that process of who
communicates and what publish, what collaboration from
what institution and where does the flow of
new ideas start to now come up from the minds
of people into broader adoption and
inspiring a whole fields and domains and that’s another
example where you can now use that aggregate a signal for many many different places
to publications, the fields of studies
and start to create that ontology that gives
me predictive power on the impact of new articles or new fields is at a hot fields, is that are
going to gain traction. These are really
that model that now takes a life of its own
that you can now extend to other areas
and in fact it’s not just the public domain, it’s also in enterprises. So in the case of
Office he would say, “Hey Office is just a bunch
of words Outlook email, SharePoint but in fact, Outlook is in fact
at this core of how people in a company interact and communicate
and collaborate. So that flow of
signal allows us to start creating
a model digital work. How collaboration happens
on the on what topic? How does people an
organization impact that? How did we now understand
the activities that people do? How do we now understand in fact the topics and
the customers and the interactions to predict effectiveness of
engagements with companies, collaborating across
teams, the likelihood of information progress that are usually scattered in
unstructured way. So we have massive systems where we’ve taken and in some
sense that journey from Bing and that technology understanding and
modeling capabilities and we brought it to office and we’re still at
the beginning of this journey, but this is a case where now
we’re saying, “Hey maybe, maybe office is in
fact representing a deeper understanding
of how work is done.” Digital work and can that create more delightful experiences
but also can that now be bolstered to help companies transform in the way that they interact with
their employees and customers. So these are just examples where knowledge starts out from, “Hey let’s make an application smarter” to starting to create data assets that can now inspire and connect
to new scenarios. Now, once you have knowledge, doesn’t mean that it’s
readily accessible by people. Knowledge is semantically
organized and understandable but
human’s ability to interact and engage and to conflate
is in fact unstructured. Let’s take for example,
search and question, Q&A. You can ask in any natural
language anything but you have to somehow map it to what’s known and modeled
in the knowledge. Well, that requires technology. That requires
understanding, both how to model language but also the
information knowledge within. You can also look at
enrichment recommendations, the relatedness of concepts so that you can start to
now connect the dots. In the worldview that
we have at Microsoft, that Satya talked
about which is this Intelligent Cloud,
Intelligent Edge. We talked about this ability
to connect the dots. It’s not just
disparate interactions with different devices. It’s a multi-device,
multi-sensory coordinated world. That coordination,
that glue is in fact that connected fabric
of people, of things, of the environments, of the
context that is the glue that starts to tie
the different pieces together into something coherent. I want to now give
some examples of the power of these experiences
when you have knowledge. This is in fact the thing
that gets us excited but again we’re at
the early stages of it. Nice animation. Let’s
go with the Bing. Starting with Bing. Clearly,
we have the ability to have reasonable and
interesting questions around the knowledge graph. So, in this case, what’s
the size of Switzerland, there’s the knowledge graph, there different nodes,
there different facets. Based on that, we can now derive the computations to answer that, in a much more directed fashion. You can imagine this across
a large number of domains. But we’ve also recognized that that doesn’t capture
all the information. That there’s still
lots of information in the billions and
billions of documents. That may not be
explicitly represented. So, there’s lots of advances in machine reading comprehension, newer modeling deep Q&A and these are things that
we have in Bing and also in the Milieu bug as well as many academia and research
systems that are out there. Now we understand
semantically model, the knowledge is within snippets, different documents,
warranties and be able to now have
Q&A around it. In fact, we’ve taken it
one step further and say, it’s not just one answer and in fact there are
different perspectives, because when you start
looking at information, it is not the facts
only, it is opinions, it’s perspectives and
the ability to surface that and to recognize that
there is also critical. So, we have this
multi-perspective answers. These are just
examples that now go beyond the ranking of
the template links to now understanding what is
the inherent intent and the need that people may have and
how do we start to surface that knowledge
is within the web. A lot of that requires us to now invest in technologies
that now to elevate and goes beyond
the indexing the posting list and to
start to look at the organization of information and to be able to
reason around it. In office, it turns out we’re also starting to now
bring that technology and that knowledge infusion into the experiences
you all know like Word. In this case, imagine you’re
writing some article, you’re writing a paper
and you want to know, hey, I got to be inspired or tell me a little bit of
contextual information. In this case, since we’re systematically bringing
what’s available on the web, from the knowledge graphs, from internal to the company
to now be contextually relevant to the things that’s happening in your work space, in the thing that you’re
currently working on. Again, it’s a different way of bringing in
that connective tissue of contexts that we think
allows you to stay in flow. If we look at productivity, one way we defined productivity
is staying in the flow, where you are most
productive as long as you can and bringing
that contextual knowledge in a relevant integrate away to
anticipate that next step is in fact a hallmark of
good knowledge capabilities. In fact, it doesn’t even stop
at the word or the flow. Imagine you’re in Excel, you got lots of different ways of describing data and values and the fact that there’s
information there are referenced that may be valuable
for you to contextualize. So, in this case for example, you can imagine that let’s
say you have the word United, but appears in
different contexts. This is just to
illustrate the complexity and the richness of the language
but also the ambiguity. So, in this case, United in the last context is in
fact a bunch of movies. But we wouldn’t know
that unless we actually understood the other elements
be able to conflate. Likewise, in other contexts
it could be an airline, it could be companies, it could be European
football clubs, I mean it could be
many different ways. But again context matters and the ability to model
that is in fact rich. So, we’re only
beginning to scratch the surface of bringing
information and knowledge in the context of user need in context or through
explicit queries and intent. But even beyond this, the one thing that
we’ve discovered as we start working on agents and assistance and
bots is that, whereas, knowledge may be considered to be valuable but it’s
still nice to have, you can still get your work
done with temporal links, you can still get
your Word document done without being inspired
by contextual search. But in the case of conversations, especially conversations that go multi-turn, you need knowledge. The reason is the context and the information interactions that you have one turn needs to be passed and transferred to the next one so that
you can start to reason in that fluidness and that sharing of
contexts across turns, where the turns and the actions
taken on each turn, may be from different vendors, different applications and
different knowledge bases, is in fact one of
the hallmark challenges and opportunities in
conversation design. So, in this case,
imagine that you are now going through this flow. In fact, this is inspired by our semantic machines
acquisition. We’re very excited to get Dan Klein and Percy to
be part of our family, but in this case,
imagine you say, “Hey, I want to go
to New York two days before thanks giving, what is two days
before thanks giving? What does it mean on New York? Where are the airports? How do I understand it? How do I reflected? How do I understand the
fastest elaboration?” These are all things that
you would say, yeah, it actually requires
you to orchestrate across many backend
systems in APIs. Each one has an ontology in some sense of the values
of the capabilities. So, in this case,
it’s really about this ability to use knowledge to connect the dots across turns to be able to reason
and contextualize and to guide that discussion. So, in this case, take travel in New York to be able
to now recognize that nearby airports of New York are JFK and this certain location. Likewise you can
now imagine each of the APIs and each of
the systems having its own data and we need
to learn the association and do dynamic conflation
across these. Now, with all of the things
said, with knowledge, there are many things you
can do to start to now bring that knowledge in the context
of interactions and flows. But how do we bring this to life? This is a rich area of
research and product efforts. I’m not going to go through the details on this because I think you’re all world experts. But I do want to
share a little bit of how we’re looking at the dimensions of what
knowledge systems, the quality. What does that mean
and also some of the challenges that
we have that we think are really pushing us
to the limit but certainly an area that we’d like to invite your creativity and
your collaboration. So, in this case, to bring
knowledge systems to live, we start with the
fact that data can be chaotic and come from
structured and unstructured. Lots of effort across different pieces but
bringing that together, in some coherent knowledge
production process to be something where is fresh, highly structured, semantically understandable,
that’s a challenge. That’s the data chaos to the structured semantics flow. There are different approaches. Not that one approach is
better but in fact over time, a number of these
will work in concert. Starting with people, Wikipedia, DBpedia like there are
certain efforts that really do require
the domain experts to start capturing that knowledge in
some descendible way and to be able to now create an
incentive that keeps it fresh. Cool Wikipedia is
the great example almost every search engine
looks at Wikipedia and say, that’s great, let’s use that
to seed our understanding. Because that’s probably
the best articulation of the basic shareable knowledge
that people interact. That’s one of the facets that I think it’s
important for knowledge. It’s not just important
for systems to understand. It is something that needs to be understandable and
explainable to people. But beyond people, at some point, you’re going to run out of
steam because at some point, either the willingness
or the capacity of people are going to hit a limit. So, this is where systems
start to come in. This theme of systems for
AI and AI for systems, is in fact one of
the hallmarks where I think it applies beautifully
to knowledge system. In this case, we can
imagine implicit modeling. We talked about deep machine
reading comprehension. We’re still at the early stages
of it but again, this is a case where
we’re now trying to understand the shape, the language, shape of information and the
shape of the retrieval. Lots of efforts on that. But there’s also knowledge representations
that are explicit. These are the triplets,
these are RDFs, these are the different
graph structures. There, I would say we have a lots of research and lots
of production systems, being Google, Facebook, LinkedIn, there are all the
systems that are creating these knowledge
representations. Some which are proprietary,
some of which are public. In fact, a lot of
research systems psych from the early days all the way to the research systems
that exist today. This is an active area. Now, what I want to do is just talk a little bit about
regardless of the approach, there are different dimensions that we evaluate and
assess knowledge. Is a knowledge system correct? What’s the degree of correctness? What’s the degree of freshness?
What’s their coverage? There’s a standard but when
you start looking at it, they work against each other
when you hit extreme scale. In fact, that’s where
we end up which is, it all sounds good for
one million documents, it sounds good for 10 million. It works really hard if
you have hundreds of billion and that’s
the chaotic Web. It’s not just chaotic web, it’s chaotic enterprise systems
and real-life situations. With that, let me just
give some examples. In the case of correctness, precision really does matter. Because once you
create that knowledge, you have to stand by the quality. If something is wrong,
you can’t just point to the source and say, I don’t know. I think I extracted it right. This is fake news. It’s
fake knowledge graph. While you got to
look at authority. You got to look at authority. You got to look at the synthesis. How do I look at voting? If I don’t have an authority,
how do I judge? How do I get user feedback? That’s a challenge
especially when you have an orchestrated multi-party
sources of information. Freshness, speed matters,
things are constantly changing. In many cases we don’t
even know what changed, not even to mention
the ability to propagate that update
through the system in a way where we understand that some of the updates may
in fact be incorrect. So, you can actually
start propagating lots of challenges
with one mistake. You can ripple and destroy pretty much everything and that’s
the lesson we have. In coverage, right size
really does matter, at some point they’ll say great, this all sounds good
but is it complete, does it capture that domain. So, all of these work
in different forces and there are lots of systems
and lots of efforts. They’re in fact at
the frontier of this. I want to list out
some of these to both acknowledge that
lots of problems still remain unsolved
but also the importance of these problems
for us to get ahead in this world where I think
knowledge is increasingly important like unsupervised,
unsupervised, autonomous. These are all keywords basically we say we can’t put humans and depend on humans to do the final verification
in all the cases. How do I unstructure? When I look at a webpage, it’s not just the head
sites that I can create templates and do
wrapper induction, site understanding, like can I understand the unstructured
nature of information? Can I understand and cluster? Can I extract facts? Can I test that hypothesis through verification
and validation? That’s a whole body of
work that pushes from the head to in fact
that torso tail. Knowledge and semantic embedding. You got lots of today explicit knowledge that’s
present today through years and decades of human aggregation or capabilities aggregation
and yet we’re building neural networks
and how do we seed it? How do we anchor
that knowledge so that you build upon it as opposed
to restart from scratch? I’ll just pick another one
like multilingual. Language fundamentally is multilingual and in
fact information on knowledge may in fact be
multilingual and multicultural. How do you even represent
that in the way in which you don’t get too precise because
once you get too precise, you get too brittle and there is no generalization all the way to you’re overly bucketing things and you’re like it’s
not that useful. So, these are lots of
active research both within technology
product companies in research as well as in
Academia I encourage continued invasion
and we will look to partner in any and all of these. But let’s share a little bit
of our learnings. There are many things we did throughout all the systems
that you heard about but I’ll just pick
a couple that I think are really hard problems and we’ve taken steps towards it
but I think it’s not a problem that’s going
to easily be solved. But I’ll just use
that hopefully to tee up the areas where I think there’s productive
research should be done. One is the inherent complexity
of the real world. We talked a little bit about it, I will share some of
the learnings that we had in addressing that. The second is this symbolic versus neural approach where you can get the best
of both worlds, both the ability to now
explore but also anchor on the knowledge that
exist and the ability to make it understandable
explainable. The last one in fact is
something that we’re seeing a huge swell of interest which is how enterprises constructed take control
to understand their unstructured in their
digital information assets in a way that makes it available
for them to change the way they interact with
customers and employees. So, I’ll just list out some, each one has its own challenges, we’ve made some progress
for it but I wouldn’t say that we’ve kind of
cracked the nut. So, let’s start with
the inherent complexity just to motivate
this a little bit. Working Web search is beautiful. It’s chaotic, nobody has
control over anything and things completely changed
at some break neck pace. But there are three dimensions
that really put pressure on the things we do. One is just the logic
which is hey, lots of detailed information describing different ontologies
across different sites, at what point does it
make sense to generalize, at what point does it become common patterns or
across domains? How do I now enlist
domain experts and how do I now recognize that domain experts
don’t exist only in one company that in fact it’s decentralized
and distributed? So, being able to tap
into that and be able to reason on both the domain specific but also
the domain general, these are the common sense. These are the basic
things on units, on basic understanding of
distance but there are lots of things that I think are shareable and fundamental. On the time dimension
everything changes, which exist that nobody tells
us that they are changing. So, even coming up with
both the ability to detect the status
updates but also be able to see whether
you can create even the incentives where
they are willing to tell you that things are changing and so to look at systems
or system integrations. In the space you can imagine
that the category and the domain and the scope continues to expand
across domains, across facets, across entity. So, in Bing, I’d say that there are
a couple of pivotal points in which we’ve shifted that really grew our horsepower
to tackle some of these. I’ll list three of
them. The first one is a shift from
batch-based conflation to a streaming-based conflation. That’s a fundamental shift. Imagine it’s no longer that hey, on Monday of every week you
take all the data sources, you do a big job and you publish a big blob of
a graph, is it correct? I hope so but it takes time, it’s used to be
a couple of weeks. But the Web doesn’t
work like that, it’s it’s not a discretized
sequence events. So, we shifted into a streaming-based system
which changes everything, the notion of having
an incremental base, the ability to start looking at updates in a different way, they are ready to now look at the incremental conflation
and evolution and even reason about
correctness in a different way. The second shift we have
is to start to look at going from the head
to the tail and that requires us to dramatically scale
our ability to both manage the ontology but also
start to go into deeper site in domain
understanding in an automated way. Again, that’s an area where
we’re pushing more and more but clearly lots
more to be done. I think the last one
that I think it’s notable is our view
of correctness. Correctness, it’s not like
everything out there is correct or in fact there are different shades
of correctness, there are different
competence factors and yet nobody is there to
be the ultimate arbiter. How do we deal with that? How do we now make sure that
inputs and changes that are proposed are hypothesis that
allows us to reason score, understand the
likelihood of churn before it goes into
the rest of the system, may be able to now
get that feedback? So, these are
just examples in which the current system continuously evolving that we like to say the colleague Yushin Gale
who’s in the audience, who’s running
this authority would say that knowledge is
always the life system, it’s a live system. It’s not like you build
it and it’s done, it’s a constant,
constant evolution. So, lots of challenges
that you can imagine both systems and
algorithmic advances that allows us to now
tackle some of these, so that they can continue to grow and evolve with
the complexity of the Web. The second area is something
we’ve talked about already, which is knowledge can be represented explicitly
or implicitly and we’ve seen great advances and great promise in the neural
modeling of information. We’ve also seen the advances on symbolic but there
are pros and cons. At some point the
discretized representation is probably too big and too hard to manage in
the context of now joining against their intent
or contexts and vice versa. So, their benefits are both.
Their benefits are both. One is understandable to humans but really
computationally not efficient when you start
looking at the mirror add up understanding that
relatedness and all that stuff, and yet our neural it’s efficient but sometimes you’re not
quite sure what’s happening. So, we’ve taken some efforts, I just list out
one example but this is clearly an active area
of research that we hope that we’ll see some good advances and
breakthrough because were seeing the need to have actually both techniques applied
systematically. One example is let’s
say you do have a neural system that
allows you to model against some embedding space and you have
a knowledge graph and the question is how do I now use a knowledge graph to
bootstrap by understanding? The way you do that is in
order to do this mapping, we first understand
the hypothesis that are available in terms of the questions and answers
in the knowledge graph. We apply that through
the system so it learns and maps it against
an embedding space. Oh sorry, I went too fast maps it against
an embedded space. So, overtime you can imagine that through techniques
like this or similar, we can hopefully start to now bootstrap and connect and
hopefully even do this in a much more integrated
way but this is an active area of
research that I encourage all of you to look at and
see what we can do together. Then on the enterprise, going back again on
this enthusiasm that people have around
enterprises and their knowledge but lots of unstructured information in
people’s communications, in the documents people write. People don’t writing structured facets and fields and certainly they write it with different
variant degrees of quality. If there’s a study from
Gartner that says, “Over 80 percent of
enterprises knowledge is locked and unavailable
for broader use.” That’s a staggering number
you talk to IT, you talk to the companies
they’re saying you know I don’t know like it’s all black-box thingy and
the data continues to grow. So, in the world in which
data is growing massively, they’re seeing
tremendous value but they can’t make sense of it because they’re stored in different ways, written in different formats and not conflated,
what do they do? That’s a real challenge but that’s also a real opportunity. So, let me give
you one example of how this is coming to bear. So, around late last year
and early this year we engage with Publicis which
is a advertising agency. They have lots of agencies they acquired like 1,200 acquisitions, they have 200 sub agencies, they have 80,000 people each one. You can imagine having
their understanding of clients. Their understanding of the work, the advertising campaigns, their deep understanding of
brands, their talent profile. So, it’s lots of information. So, in their case,
what they’re saying is they want to see
the benefits of scale. Here’s an example of
a digital transformation where companies really at
the very fundamental level which is the fact that I’ve 80,000 and 1,200 acquisitions how do
I get skill advantage? Well, to get skill
you got to bring that ring formation and
that people asset together. So, what we were working
with them is to start to bring together their
understanding of their talent, understanding of their accounts, understanding of the work
they’ve done and start to create really this ability to now reason, map and correlate. This is something that
I’d say that if you’re an online company or
a commerce company, you just do this but for enterprise this is all new stuff. This is all new stuff
and in fact it requires a deeper understanding
appreciation of how do you reason with knowledge
and to get that flying. But in the course of
these engagements, we also recognize that
enterprise knowledge is unique. It’s unique it’s
got its own unique challenges in two dimensions. One is, data in enterprises is not readily
accessible to everybody, it’s not like it’s
a public web documents that anybody can read. This is private email, right? This is documents
that are sensitive, this is Lean or ACP documents. There are lots of inflammation that you just cannot even see and enterprises have obligations both in terms of
regulatory compliance, right? Certain financial data,
certain HR data can’t be shared. There’s certainly
encumbrance, commitments. If you get data from
third parties for whatever reason you have obligations to limit
its scope of use. You have security considerations, different people have
different access rights and you have privacy with GDPR. That ability for end users both employees and
customers to now have control and that ability to influence that data
is critical. So, all this is creating a case where in many cases
we’re not quite sure how do you build
an AI model where you can’t reliably see the data. That’s a real challenge, so within Microsoft Office
for example, we have different
practices we keep out the highest standard and we in fact enlist the users
in this case, Microsoft employees to
contribute the portion their email for us
to build the models. But you can imagine
new techniques and new approaches are
needed for us to get beyond this hump and that is not necessarily
the same algorithms and approach that works
in the web and in public data can work in
the enterprise setting. So, lots of innovations around multi-party
secure computations, private computations
that are potential here. But perhaps the
most challenging is that their access
to the data within an enterprise is of mixed degrees of quality
and completeness, right? It’s a bootstrapping problem. Hey, I’m interested
in doing all this but the data may not be clean. It has the variant
degrees of quality. It may not be complete in fact, in order to bring
that domain expertise it’s typically diffused. It’s not any one organization has that depth and
certainly not the same the talent that knows the domain doesn’t necessarily
understand the systems. So, with that I would
say we’re at the stage where we are seeing
this desire to transform and knowledge
is increasingly this fundamental capability
that is starting to reshape the way we think about information
and interactions. So, I would just say that imagine fast-forward a couple years
from now where you’re saying imagine you’re living in
the knowledge rich world. Every person has
behind him or her, this rich knowledge graph of their activities interests,
interactions, relationships. Every object,
every physical space has its own knowledge graph
that’s created from different organizations
for different purposes, and every service has
its own associated knowledge. Bringing all that together in some coherent way where they’re
multi-party they aren’t working together and yet you need to now a test
and navigate and interact in a fluid way across is a real opportunity but
it’s a real challenge. So, with that these are some of the questions and domains, and it is hopefully there to
inspire that I believe we are heading in a way where we are going to see
a knowledge ecosystem. We’re going to see
knowledge technologies for both the production and
the consumption of knowledge. It will require different teams and different people to work together and we’re still early in this research and
technology innovation wait. So, with that let me
bring in my colleague Kuansan to come talk to us about one example of how we want
to push on this process.>>Great thank you. Thank you David. Good morning. It is my pleasure to be here to talk to you about
knowledge system and AI. After all, it was eight years
ago in this very venue, Microsoft Research
Faculty Summit in 2010, that we first described our ambition to teach the machine to acquire knowledge
from the web by itself. In the ensuing years
in addition to the continuing investment from Microsoft that David
has just described. We’re glad to see that the idea, the little idea presented eight years ago has received
industry-wide adoption. Including the Google’s knowledge graph efforts two years after our faculty summit and the
Baidu’s announcement in 2014. So, today it is
my pleasure to be here to share with you
the next evolution for us to share the
resources we have from the industry research lab
that consists of a data set and open source tools that can facilitate research
with the hope that more of you can join us
in a journey to advance the state of art of a knowledge systems
and AI research. So, let me start
with the data set. This is as David alluded, Microsoft Academic Graph like
the Bing knowledge graph it is build by extracting the
larger from the entire we. So, we have the scale to cover all the scholarly communications published in the past one
and a half centuries. So, speaking of ecosystem, this is knowledge graph
that every one of you is already on as a node so is every student
that you have supervised, every institution that
has sponsored your work, all the journals that
you have cited and the conferences that you have
gone to present your work. So, as you see, the number
is increasing very- is massive and it’s
growing rapidly. Right. So, one of the benefit of sitting on top of Bing is we have
the broad coverage. So, as you can see, that we have teach auto machine to go beyond computer science, and to cover more than 200,000 fields and sub-fields
including medicine, art, history, and so on. This broad coverage
turns out to be very important for Research Managers,
and your Provost, and Deans maybe, to understand the broader impacts
of our research, and for many decision-makers to determine the
investment or research. So, I hope that this gives you some incentive to take a look
at the data, but for us, it gives us a strong incentive
to push the technology, to make sure the data
quality is here. Being Researches ourselves, we naturally have published
paper to describe how this graph is created and
why we envision it can do. The easiest way to
find this paper is to remember it is
published in www 2015. So, in this system, you can just type www
2015, and right there, you can see all the papers
are published in that year, and our paper is actually
coding rank number two of the search results. So, they are choosing
immediately as you can notice. The first is the query. So, by recognizing
the conference as a first-class citizen
in the knowledge graph, we can actually do more better search experience by not just keyword matching
the query terms to the title. I understand that many of my colleagues have been using
this feature when they are running the test of
time awards committee to find papers and their citations
afford particular venue, and these two, you can do it with a single query
in this tool. I’m also glad to report
for the past two years since I discovered this new
usage, I’ve been watching it, and ours ranking system
has been able to predict the test of time awards
winners quite accurately, and that’s actually
the second thing that I would like to talk about, the ranking here is actually the full
blown search ranking, and it’s not just based
on citation counts. As you can see,
the third search results actually received
more citations than us. But the ranking algorithm here is actually also estimating the reputation of
the sighting parties and not just politely
kind of citation. So, I’ll be more than happy to talk more about it
with you afterwards. So, if this trend continues, I’m predicting that www 2025, this paper about
a neural representation of knowledge system is likely
the winner of the test of time awards and this paper
is that coincidentally is from my colleague
in MSRA gene town. Now, for many of you,
speaking with Jian, who unfortunately to have is a very common name and many of you to have
the common name, to get your research results are aggregated correctly on the search engine has been
a challenge, isn’t it? So, let’s write it for
my colleague Jian. If you tried with your favorite search engine
keyword-based search engine, you will see, well, yes, there are quite a few
Jian Tang in the system and the results are actually
intertwined together. But because the system is
sitting on top of a Bing, we were able to use our all sorts of knowledge including your CVs, and resumes published on the web, and who actually learn how to disambiguate
different authors. So, for example,
when in this system, we actually when you hovered
over different names, internally at the backend, we actually understand
which Jian Tang is which and not to mention all sorts
of other variation, our physics colleagues
like to publish their papers with only first
initial and last name, and in many publication, they actually observe
the Asian convention by putting family first
rather than last. So, these are old or
different hypothesis we put it here as I alluded
as your racing, you can just hover over it. In this case, this is
our colleague Jian what I want and just add
one additional colleague. Voilà. All his work is aggregate correctly
and showing here. I hope this is to give you a quick understanding about what the knowledge system can
help us and how you can enrich the experience we have. So, let’s now switch
back to the PowerPoint. So, if you are interested, find this useful or interested, these are the resources. We’re making freely
available to everyone. The first one is the graph, the dataset is available
through Azure. Again, freely
available to everyone. So, in the same folder, you will find a lot
of open source tools. Hopefully, can
actually jump-start all these operations to deal with large data sets and
create knowledge systems. We encourage you to take
a look and as you see, we have already more
than 127 systems that are already published
based on this dataset. I invite you to take a look and tell us
what you can build. More information, here’s
the URL, With that, thank you for your attention and let
me welcome David back.>>All right. Thank you.
Okay. Let me wrap it up. If you roll back
at the beginning, what we’re seeing is this emergence of
an appreciation for knowledge. Knowledge is going to push
the limits of systems and AI, but it has a transformative
effect in the way we think about products and
we think about companies. This is also rich area
where we’re still at the forefront of
that technology race in that technology push. With that, I want to really thank all of you for your attention, your participation in
the faculty summit. I’m excited that Microsoft is such a strong advocate for
academia and for research. We are looking forward to seeing amazing things from all of you and from our
collaboration. So, thank you.

Leave a Reply