I’ve just discovered an incredibly cool project on Github: DeOldify, which uses deep learning to automatically colorize old black & white photos. It’s not perfect, but what it’s able to do is pretty amazing, and improving rapidly.
In addition to ninja-level coding, author Jason Antic (@citnaj on Twitter) does a terrific job writing up how the algorithm works in the README file.
Essentially, his code uses a deep learning technique called a Generative Adversarial Network (GAN.) GANs consist of two components: a “Generator” and a “Discriminator.” In brief, the Generator (its own neural network) attempts to synthesize fakes or originals, and the Discriminator attempts to figure out if the submitted instance is “real” or fake. In this way, a Generator can be considered a “Counterfeiter” trying to fool the Discriminator, and the Discriminator can be thought of as “The Police” trying to catch the counterfeiter passing off a fake.
A zero-sum game then is played over and over, thousands of times, with each party trying to maximize its winnings, within some constraints.
Over time, the counterfeiter gets better and better at counterfeiting, until it can stand on its own and truly generate something that is pretty close to good output. To me, this is somewhat analogous to how children learn to better tell the truth when they experiment with fanciful lies in pre-adolescence. (Some, sadly, never learn the lesson fully.)
Back to DeOldify. The input/output is pretty impressive, with several examples provided at the link above.
Examples
I went through the Google Colab (Jupyter Notebook) powered harness developed by Matt Robinson, and input a few photos. Keep in mind that none of this involved any Photoshop work on my end:
Not perfect, but remarkable nonetheless!
GANs are perhaps the most interesting thing coming out of the work in Machine Learning these days.
I plan to spend some time with this project and see how well it does on some family photos from decades past, and perhaps build a web front-end to make it easy to try out. (It’s computationally expensive, so perhaps that front-end will need to rely upon donations.)
UPDATE: Jason Antic continues to make incredible progress on the model. The new images are much better than those above. Give it a whirl at https://deoldify.ai/!
Steve’s an entrepreneur and software leader. Most recently, he founded HipHip.app, the easiest way to create celebration videos. He also founded bigthanks.org, helping people discover and share productive ways they can respond in times of crisis. Steve’s worked on consumer apps, online travel, games, relational databases, management consulting and telecom. He launched Alignvote in 2019, which helped Seattle voters find their best-match political candidates. Steve founded BigOven, the first recipe app for iPhone, now with more than 15 million downloads, which was purchased in 2018. Steve served as Chairman of Escapia Inc., the leading SaaS solution for the US vacation rental industry, sold to Homeaway, now part of Expedia. In 1997, Steve was cofounder, President, CEO and Chairman of VacationSpot, a pioneer in the online reservation of vacation rentals, bought by Expedia in January 2000. At Expedia, Steve was Vice President of Vacation Packages, leading the vacation package and destination services teams, helping to create two patents on the first-ever dynamic vacation packaging system on the Internet, which now represents billions in annual transactions for Expedia.
He has keynoted on several occasions at the Vacation Rental Managers Association (VRMA), and taught a graduate level course on the strategic management of innovation at the University of Washington Foster Business School in Seattle, Washington.
Steve worked for Microsoft from 1991 to 1997 in a variety of senior marketing and executive positions, and led the creation of the internet games group, helping develop several products and patents related to online multiplayer gaming. He helped launch Microsoft Access and was involved in the acquisition of Fox Software by Microsoft in 1993. He’s worked for IBM, Booz-Allen Hamilton and Bell Communications Research.
He holds an MS in Computer Science from Stanford University in Symbolic and Heuristic Computation (AI), an MBA from Harvard Business School, where he was named a George F. Baker Scholar (awarded to top 5% of graduating class), and a dual BS in Applied Mathematics / Computer Science and Industrial Management from Carnegie Mellon University (CMU) with University Honors. Steve volunteers when time allows with Habitat for Humanity, University District Food Bank, YMCA Seattle, Technology Access Foundation (TAF) and other organizations in Seattle.
