Keynote: When Machine Learning Can't Replace the Human

Location: Grand Ballroom ABC & Simulcast in Garden

Duration: 5:30pm - 6:15pm

Day of week:

This keynote is now available to view on

Watch video with transcript


In our modern era of digital cameras and automated imaging, hoards of data are piling up that can be used to solve a myriad of problems: but only if there is a way to categorise and tag the images. Machine Learning is making great strides, but despite all attempts to make it the hammer that bends all nails, there are still problems that require human vision. In this talk, we will look at the case of an asteroid - a 500m across rock named Bennu. While unimpressive in size, this orbiting rubble pile has posed an intimidating challenge to its mission team: How can a safe spot to get a sample (or 3) be found quickly on an object with half a million hazards (give or take). The answer? Use humans as part of the algorithm. This solution presents its own problems, ranging from griefers to slackers to ethical considerations, but this solution works. Come for the computer science, and stay for the astronomy, as we explore how creative software solutions let scientists explore our solar system.


What is the work that you're doing today?


I'm working to find ways to help us use technology to map other worlds. At this particular moment in time, computer vision isn't quite ready to mark the hazards our spacecraft face day today. I'm trying to figure out how to integrate in humans in my algorithm and do it in a way that's ethical and can, where possible, feed into training software to someday take over in the future.


What are your goals for the talk, what are you hoping that the audience might take away from listening to you? 


If nothing else happens, I want people to stop automatically saying "why don't you just apply machine learning to that?" Because there are so many problems that machine learning still can't solve. It's so important for us to realize both the limits of our technology and the fact that we have other tools in our toolset. Let's remember to use that screwdriver for the screws and reserve the hammer for the nails.

One of the things that I'm really hoping to get into is how we have lots of niche problems that may only have a thousand images, a few hundred plots. For these particular datasets, we need to be able to dig through them and find the patterns, find the trends. The human mind is still the most powerful tool we have to do that. Where just like we sometimes use open source to solve our software limits while we need to use crowdsourcing to solve our computer vision limits.


What do you think is the next big disruption in software or technology?


One of the biggest issues facing so many problems right now from self-driving cars to the algorithms that suggest our next video on YouTube, the next TV show on Netflix, is figuring out how to deal with the gray areas, how to deal with the multiple possible solutions. When software starts to be able to figure out how to make ethical choices, we're going to start to be in the science fiction future we're not quite ready for. And we're getting really close. 

It starts to become a question of how far are we from not being able to separate out our algorithms from our colleagues. When that happens, it's going to lead to a societal upheaval that will be awesome in terms of having robotic caretakers in places like Japan where there just aren't enough in that next generation, but also awful in the places where we do still have enough humans to fill a lower level workforce jobs. What do we do when we disrupt our entire economy while just solving our software issues.

Speaker: Pamela Gay

Senior Scientist @planetarysci (Planetary Science Institute)

Dr Pamela L. Gay is an astronomer, technologist, and creative focused on using new media to communicate astronomy and space science. She is co-host of the award-winning Astronomy Cast podcast. Through this show, she and co-host Fraser Cain take their audience on a fact-based journey through the universe, exploring not just what we know but how we know it.  

In addition to teaching astronomy through podcasts, Dr Gay is Director of CosmoQuest, an online facility for learning and doing astronomy. In spring of 2019, CosmoQuest community members will play a significant role in the OSIRIS-REx mission by helping map this asteroids surface. As part of CosmoQuest, she also hosts the Daily Space, a news show on Twitch and YouTube.  

While she is most well known for her public-facing science communications efforts, behind the scenes, Dr. Gay is a programer innovating how citizen science can advance science. From experimenting with UX to experimenting with machine learning, her team combines big data number crunching and web development to get at new understandings of the Universe.  

Dr. Gay is a senior scientist at the Planetary Science Institute, a 2018 Podcasting Hall of Fame inductee, and recipient of the 2019 Isaac Asimov Science Award. She can be seen on television on shows like Strange Evidence and The Universe, her writing has appeared in Sky & Telescope, Lightspeed, and other magazines, and she regularly consults on science fiction novels.  

In addition to her academic work, Dr Gay is also a space artist who turns the latest news in planetary science and turns it into artwork and poetry. She has sold more than 100 original paintings to personal collectors, coffee shops, and everything in-between.

Find Pamela Gay at

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.