Data Critique – Machine Learning

For my Machine Learning for the Web final I decided to build a game in P5 using StyleGan, a model that generates high quality synthetic images of human faces. The game is pretty simple, detect the real human photo from artificial images.

Early prototype

StyleGAN gives control over the style of generated synthetic images by modifying the input of each level separately. Generating the artificial image gradually at a low resolution and finally in high resolution allows for better control of facial features and fine details without dramatically affecting other levels.

Initially, I used Flickr Faces for my model set up in a program called RunwayML. Here are the results:

Notice the pattern? Where are the black and brown faces? I did a basic Google search for the following:

NYC train station
People in SOHO
People walking in NYC

NYC Train Station
People in SOHO
People walking in NYC

I even tried ” People in America”.

People in America

Clearly, my search was accurate – for the most part, I’m surrounded by people who look like me — daily. Unfortunately, StyleGan doesn’t populate artificial images of black and brown people as frequently as non black and brown people.

Why I am so grossly underrepresented in StyleGan? Let’s take this website for example. Its absolutely possible to go through a dozen images without seeing one black person. In fact, I was able to do this several times over.

46 StyleGan images later (original StyleGan)

Note: StyleGan just released a new version as of December 2019 which improves the overall image quality of generative image modeling. I was able to find the generated image below after refreshing StyleGan2 31 times.

Possible solution: Train new model with more diverse images

  1. Web scrapping – use Flickr API to create a script using tags to fetch 5000 images of black people (confirm if this is legal)
  2. Upload the dataset to RunwayML
  3. Use StyleGan Flickr Faces pre-trained model for dataset

Representation is critical – If this works as planned, my model will generate high quality diverse realistic StyleGan generated images.

One Comment

  1. Evy says:

    I love it!!!

Comments are closed.