Tag Archives: deep learning

Turkeys + Dinner Plates = Thanksgiving: Google Tries to Make Machine Learning a Little More Human

by

From my story in MIT Technology Review:

Google CEO Sundar Pichai told investors last month that advances in machine-learning technology would soon have an impact every product or service the company works on. “We are rethinking everything we are doing,” he said.

Part of that push to make its services smarter involves rethinking the way it’s employing machine learning, which enables computers to learn on their own from data. In short, Google is working to teach those systems to be a little more human.

Google discussed some of those efforts at a briefing Tuesday at its headquarters in Mountain View, California. “We’re at the Commander Data stage,” staff research engineer Pete Warden said in a reference to the emotionless android in the television show Star Trek: The Next Generation. “But we’re trying to get a bit more Counselor Troi into the system”—the starship Enterprise’s empathetic counselor. …

Read the full story.

Behind The Would-Be Siri Killer Facebook M, A Battle Over AI’s Future

by
Facebook M

Facebook M

From my Forbes blog:

Facebook’s test release today of a digital assistant inside its Messenger app is a shot across the bow of the Internet’s biggest companies: Apple, Google, Microsoft, and Amazon.com. It’s also the latest salvo in a high-stakes battle over the ways artificial intelligence should transform the way we live and work.

Facebook M is intended to allow users of Facebook Messenger to pose any query or service request in natural language and get a personalized answer immediately. The key wrinkle that sets it apart from Apple’s Siri, Google Now, and Microsoft Cortana is that there’s a team of human “trainers” who will step in when the machines aren’t quite up to the challenge.

So far, it’s only available to a few hundred people in the San Francisco Bay Area, and its timing and scope are unclear. But judging from a brief post by VP of Messaging Products David Marcus, Facebook M is clearly a major bid in a quickening battle to be the virtual assistant of choice, taking on not only Siri, Google Now, and Cortana, but also a raft of upstarts such as Luka, Magic, and Operator.

And in the mobile age, virtual assistants could prove to be the key product that will define which companies dominate the next decade of online services, just as search was for the past decade. “Whoever creates the intelligent assistant will be the first place people go to find things, buy things, and everything else,” former AI researcher Tim Tuttle, CEO of the voice interface firm Expect Labs, said last week.

But what’s even more interesting in the bigger picture is how Facebook M plays into a longstanding, fundamental battle over how artificial intelligence should be employed–one that has recently come into sharper focus. … The upshot: Until and unless AI gets so good that machines can anticipate what we want, people will remain a key component of truly intelligent online services.

Read the entire post.

Why Are You Still Typing On Your Phone – Or Any Other Device?

by

From my Forbes post:

Only a few years ago, you thought that guy walking down the street apparently talking to himself was off his meds. Now, you’re rocking your Bluetooth headset every day without even thinking about it (even if you still annoy some of us on the train).

But that’s talking with other people, for pete’s sake–are you still phoning with your phone in 2015? Today, you can ask it to do almost anything just by speaking “OK Google” or “Hey Siri”: conduct a search, make a restaurant reservation, send a text, or do almost anything you used to have to type into a search box or tap into an app.

You probably already knew you could try doing all that, but here’s what you may not know: Most of it doesn’t suck anymore. If you haven’t tried Google Voice Search, Apple’s Siri, Microsoft’s Cortana, or even Amazon.com’s Echo “smart” speaker recently, you may be surprised how much better they’re working than even six months ago. Not only do they seem to understand words better, even in noisy situations, they also appear to produce more accurate results in many cases.

All that’s thanks to big improvements in machine learning, in particular a branch of artificial intelligence called deep learning, that’s been applied to speech recognition in the last couple of years. “Recent AI breakthroughs have cracked the code on voice, which is approaching 90% as good as human accuracy,” says Tim Tuttle, CEO of Expect Labs, which began offering its MindMeld cloud-based service last year to help any device or app create voice interfaces.

It’s great for us smartphone owners, but the stakes couldn’t be higher for companies. …

Read the full story.

With IdeaMarket, Idealab’s Bill Gross Wants To Create 1 Million Startups

by

From my Forbes blog:

You might wonder if perhaps there are a few too many startups these days, especially if you’re trying to rent a place in San Francisco or buy a house in Palo Alto. Bill Gross doesn’t–not one bit.

Gross’ Los Angeles tech startup incubator Idealab has created more than 125 since its founding in 1996, 40 of them making it to IPO or acquisition. But the company’s founder and CEO thinks he has come up with a way to multiply that sum by about 8,000, to as many as 1 million startups eventually. The new company he’s announcing this morning at the TechCrunch Disrupt startup-launching conference in San Francisco, IdeaMarket, is intended to be a startup marketplace that matches ideas with investors and especially entrepreneurs. “IdeaMarket is the culmination of my whole life,” Gross said in an interview. “It’s turning what I do into a machine.”

Something of a mashup of Kickstarter, Quirky, and XPrize, as well as Y Combinator and other incubator/accelerators, IdeaMarket will let anyone post an idea for a product or service that they don’t have the resources or desire to pursue themselves. They can invest in it, and so can other accredited investors, who may offer, say, $100,000 apiece to entrepreneurs who want to take the idea and run with it. An entrepreneurial team submits a plan for how they’d do that and the investors or IdeaMarket interview the candidates to make a choice. Visitors to the site can vote on them or suggest improvements, or even invest in them once they get accredited.

So far, prominent investors and tech figures have come up with more than 20 ideas. Listed already among 17 ideas with a combined $2.7 million in committed funding are a 3D printer than can print glasses lenses (from Index Ventures cofounder Neil Rimer); an app that tracks your app usage and puts the most-used ones at the top of your smartphone screen (from Google developer advocate Don Dodge); an Uber for trash pickup called Trashnado (from entrepreneur and angel investor Scott Banister); pizza delivery robots (from Gross himself) and (saving the strangest for last), Pray It Forward, “a web-based marketplace for people in times of trouble to quickly tap into the power of group prayer by connecting them with people who will pray for them” (from Affirm cofounder and CEO and former PayPal cofounder Max Levchin). They’re all also investors, along with others such as SherpaVentures cofounder and managing partner and former Menlo Ventures managing partner Shervin Pishevar.

While you can imagine IdeaMarket might spur yet another round of apps that we probably have too many of already, most of which will either wither or get sucked up by Google, Facebook, and the like, Gross is clearly hoping for more groundbreaking ideas as well. …

Read the rest of the story.

Interview: Inside Google Brain Founder Andrew Ng’s Plans To Transform Baidu

by
Baidu Chief Scientist Andrew Ng

Baidu Chief Scientist Andrew Ng

From my Forbes blog:

Little known outside China, the Chinese search engine Baidu scored a coup earlier this year when it hired Andrew Ng to be chief scientist and open a new artificial intelligence lab in Silicon Valley. Ng, a Stanford computer science professor who headed the Google Brain AI project and then cofounded the online education startup Coursera, is the foundation for Baidu’s plan to transform itself into a global power.

In two wide-ranging conversations at Baidu’s still mostly empty Silicon Valley Artificial Intelligence Lab in Sunnyvale, adjacent to the rocket scientists at NASA’s Ames Research Center, Ng and his lab chief Adam Coates recently outlined their plans at Baidu and their vision of what AI can accomplish. That, as I outlined in a story on those plans, includes everything from improved speech recognition to much smarter robots to truly intelligent personal assistants.

Ng, who speaks in an extraordinarily gentle voice that compels close attention by the listener, seems to realize how much he has to prove, both vs. fast-rising Chinese rivals such as soon-to-go-public Alibaba and global forces such as Google and Facebook that are also betting big on AI, in particular the fast-emerging branch of AI called deep learning. Even before being asked, Ng sought to quash what he called the “stereotype” of Chinese companies as mere copycats of U.S. and other technology companies.

In Baidu’s case, at least, the stereotype may be superficial. But it also seems clear that Ng’s hiring is part of an attempt by Baidu, often called “China’s Google,” to create world-beating technologies that will elevate it to the top tier of global innovators. In this edited version of the interview, he reveals plenty of details about how he plans to help make that happen.

Q: How did you get interested in artificial intelligence?

A: I just thought making machines intelligent was the coolest thing you could do. I had a summer internship in AI in high school, writing neural networks at National University of Singapore–early versions of deep learning algorithms. I thought it was amazing you could write software that would learn by itself and make predictions.

If we can make computers more intelligent–and I want to be careful of AI hype–and understand the world and the environment better, it can make life so much better for many of us. Just as the Industrial Revolution freed up a lot of humanity from physical drudgery, I think AI has the potential to free up humanity from a lot of the mental drudgery. …

Read the rest of the interview.

Startup Ersatz Labs Launches Deep Learning AI In The Cloud (Or In A Box)

by

From my Forbes blog:

Deep learning, a branch of artificial intelligence that has led to recent breakthroughs in automated image and speech recognition, is the hot new technology among tech giants from Google and Facebook to Microsoft and China’s Baidu. They’ve been spending hundreds of millions of dollars to buy companies and vacuum up talent from universities that are all working on deep learning neural networks, which attempt to mimic how the brain works to improve computing performance on tasks humans do with ease.

Now, a San Francisco startup called Ersatz Labs is formally launching what it calls the first deep learning platform, one that it says any company or researcher can use to do deep learning on the (relatively) cheap. It’s being offered as a service in the cloud and, for companies that want or need to do what can often be mission-critical work inside their corporate network firewall, as a hardware appliance with software installed.

Either way, the upshot of the service is that a whole lot of companies may be able to apply deep learning to their own services to achieve similar breakthroughs to Google’s, Microsoft’s, and others’. The service has been in beta for the past year with 2,200 customers, from Wall Street traders to researchers looking to detect tumors on mammograms to energy companies analyzing seismic data to an iPhone app maker using accelerometers to determine if you’re exercising correctly. “The deep learning methods are established enough that you don’t have to build this all yourself,” says Ersatz Labs CEO Dave Sullivan, cofounder with Chairman Ronjon Nag, who also provided $250,000 in funding. …

Read the rest of the story.

Meet The Guy Who Helped Google Beat Apple’s Siri

by
Google's Jeff Dean

Google’s Jeff Dean

From my Forbes blog:

For all the attention lavished on Siri, the often-clever voice-driven virtual assistant on Apple’s iPhone, Google’s mobile search app lately has impressed a lot more people. That’s partly thanks to Google Now, its own virtual assistant that’s part of that app, which some observers think is more useful than Siri.

But the success of Google’s mobile search stems at least as much from a big improvement over the past year in Google’s speech recognition efforts. That’s the result of research by legendary Google Fellow Jeff Dean and others in applying a fast-emerging branch of artificial intelligence called deep learning to recognizing speech in all its ambiguity and in noisy environments. Replacing part of Google’s speech recognition system last July with one based on deep learning cut error rates by 25% in one fell swoop.

As I wrote in a recent article on deep learning neural networks, the technology tries to emulate the way layers of neurons in the human neocortex recognize patterns and ultimately engage in what we call thinking. Improvements in mathematical formulas coupled with the rise of powerful networks of computers are helping machines get noticeably closer to humans in their ability to recognize speech and images.

Making the most of Google’s vast network of computers has been Dean’s specialty since he joined Google an almost inconceivable 14 years ago, when the company employed only 20 people. He helped create a programming tool called MapReduce that allowed software developers to process massive amounts of data across many computers, as well as BigTable, a distributed storage system that can handle millions of gigabytes of data (known in technical terms as “bazillions.”) Although conceptual breakthroughs in neural networks have a huge role in deep learning’s success, sheer computer power is what has made deep learning practical in a Big Data world.

Dean’s extreme geekitude showed in a recent interview, when he gamely tried to help me understand how deep learning works, in much more detail than most of you will ever want to know. Nonetheless, I’ll warn you that some of this edited interview still gets pretty deep, as it were. Even more than the work of Ray Kurzweil, who joined Google recently to improve the ability of computers to understand natural language, Dean’s work is focused on more basic advances in how to use smart computer and network design to make AI more effective, not on the application to advertising.

Still, Google voice search seems certain to change the way most people find things, including products. So it won’t hurt for marketers and users alike to understand a bit more about how this technology will transform marketing, which after all boils down to how to connect people with products and services they’re looking for. Here’s a deeply edited version of our conversation:

Q: What’s “deep” about deep learning?

A: “Deep” typically refers to the fact that you have many layers of neurons in neural networks. It’s been very hard to train networks with many layers. In the last five years, people have come up with techniques that allow training of networks with more layers than, say, three. So in a sense it’s trying to model how human neurons respond to stimuli.

We’re trying to model not at the detailed molecular level, but abstractly we understand there are these lower-level neurons that construct very primitive features, and as you go higher up in the network, it’s learning more and more complicated features.

Q: What has happened in the last five years to make deep learning a more widely used technique?

A: In the last few years, people have figured out how to do layer-by-layer pre-training [of the neural network]. So you can train much deeper networks than was possible before. The second thing is the use of unsupervised training, so you can actually feed it any image you have, even if you don’t know what’s in it. That really expands the set of data you can consider because now, it’s any image you get your hands on, not just one where you have a true label of what that image is [such as an image you know is a cheetah]. The third thing is just more computational power. …

Read the full interview.

Interview: How Ray Kurzweil Plans To Revolutionize Search At Google

by
Google's Ray Kurzweil (Photo: Wikipedia)

Google’s Ray Kurzweil (Photo: Wikipedia)

From my Forbes blog:

When Google announced in January that Ray Kurzweil would be joining the company, a lot of people wondered why the phenomenally accomplished entrepreneur and futurist would want to work for a large company he didn’t start.

Kurzweil’s answer: No one but Google could provide the kind of computing and engineering resources he needed to fulfill his life’s work. Ever since age 14, the 65-year-old inventor of everything from music synthesizers to speech recognition systems has aimed to create a true artificial intelligence, even going so far as to predict that machines would match human intelligence by 2029.

Now, as a director of engineering at Google, he’s focusing specifically on enabling computers to truly understand and even speak in natural language. As I outlined in a recent story on deep learning–a fast-rising branch of AI that attempts to mimic the human neocortex to recognize patterns in speech, images, and other data–Kurzweil eventually wants to help create a “cybernetic friend” that knows what you want before you do (that is, if someone else doesn’t get to it first).

Indeed, Kurzweil’s focus is timely from a competitive standpoint as well. Google upped the ante on Apr. 29 by bringing its Google Now voice search app to the iPhone and iPad, in direct competition with Apple’s Siri. And Facebook just revealed that it built a natural-language interface for its Graph Search service announced earlier this year. It’s becoming clear that search is already starting to move beyond the “caveman queries” that characterized effective search techniques until recently.

In a recent interview I conducted for the story, Kurzweil revealed a surprising amount of detail about his planned work at Google. No doubt the nature of that work will evolve as he settles in at the company, but so far, this interview provides possibly the deepest look so far at his plans.

At least initially, that work won’t relate directly to advertising. But marketers will need to understand how profoundly Kurzweil’s and others’ work at Google could change not only what search will become in the age of more and more intelligent machines, but  the way we interact with information and even each other. All that is sure to mean big changes in the nature of advertising and marketing–well before 2029.

Q: In your book, How to Create a Mind, you lay out a theory of how the brain works. Can you explain it briefly?

A: The world is hierarchical. Only mammals have a neocortex, and the neocortex evolved to provide a better understanding of the structure of the world so you can do a better job of modifying it to your needs and solving problems within a hierarchical world. We think in a hierarchical manner. Our first invention was language, and language is hierarchical.

The theory behind deep learning, which I would call hierarchical learning, is that you have a model that reflects the hierarchy in the natural phenomenon you’re trying to learn. If you don’t do that, it’s going to be much weaker and fooled by apparent ambiguities.

Q: How will you apply that theory at Google?

A: What I’ll be doing here is developing hierarchical methods specifically aimed at understanding natural language, extracting semantic meaning … actually developing a way to represent and model the semantic content of documents to do a better job of search and answering questions.

An increasing percentage of queries to Google are in the form of questions. The questions right now can’t have an indefinite complexity to them. But if we can actually model language in a hierarchical fashion, we can do a better job of answering questions and doing search in general, by actually modeling what all these billions of web pages are trying to say. …

Read the rest of the interview.

This Is How Google (And Its Advertisers) Will Really Get Inside Your Head

by

HAL9000

From my Forbes blog:

Google cofounder Sergey Brin said only half-jokingly back in 2002 that his company aimed to create the equivalent of the intelligent computer HAL 9000 in 2001: A Space Odyssey, but without the bug that resulted in it, you know, killing people.

More than a decade later, Google isn’t nearly there, for better or worse. But lately, it has been aiming much more directly at building HAL, or what’s sometimes called the Google Brain. As I wrote in a recent article, a fast-emerging branch of artificial intelligence called deep learning is helping Google and other companies and researchers produce significant advances in machines that at least approach the way we think. It won’t be long–for better or worse–before their work also has a profound impact on marketing and advertising as well. …

Read the rest of the analysis.

Deep Learning: Artificial Intelligence Finally Gets Smart

by

building.a.brain-tr

From my feature story in Technology Review:

When Ray Kurzweil met with Google CEO Larry Page last July, he wasn’t looking for a job. A respected inventor who’s become a machine-intelligence futurist, Kurzweil wanted to discuss his upcoming book How to Create a Mind. He told Page, who had read an early draft, that he wanted to start a company to develop his ideas about how to build a truly intelligent computer: one that could understand language and then make inferences and decisions on its own.

It quickly became obvious that such an effort would require nothing less than Google-scale data and computing power. “I could try to give you some access to it,” Page told Kurzweil. “But it’s going to be very difficult to do that for an independent company.” So Page suggested that Kurzweil, who had never held a job anywhere but his own companies, join Google instead. It didn’t take Kurzweil long to make up his mind: in January he started working for Google as a director of engineering. “This is the culmination of literally 50 years of my focus on artificial intelligence,” he says.

Kurzweil was attracted not just by Google’s computing resources but also by the startling progress the company has made in a branch of AI called deep learning. Deep-learning software attempts to mimic the activity in layers of neurons in the neocortex, the wrinkly 80 percent of the brain where thinking occurs. The software learns, in a very real sense, to recognize patterns in digital representations of sounds, images, and other data.

All this has normally cautious AI researchers hopeful that intelligent machines may finally escape the pages of science fiction. Indeed, machine intelligence is starting to transform everything from communications and computing to medicine, manufacturing, and transportation. The possibilities are apparent in IBM’s Jeopardy!-winning Watson computer, which uses some deep-learning techniques and is now being trained to help doctors make better decisions. Microsoft has deployed deep learning in its Windows Phone and Bing voice search. Says Peter Lee, head of Microsoft Research USA, “Deep learning has reignited some of the grand challenges in artificial intelligence.” …

Read the full story.