Derek Meer

Derek's blog

That's debatable...

Motivation

Usually, when my dad and I go hiking in the mountains and we spot a bear (from a safe distance), we have a tough time figuring out if it's a brown bear or a black bear. How you respond to a bear encounter depends on the species. For brown bears, you're supposed to pretend you're dead. For black bears, you're supposed to be loud and intimidating to scare them off. Being unable to tell the two apart could be the last mistake you ever make.

Unfortunately, telling them apart isn't as simple as the difference between black and brown, as the National Park Service points out:

The name “black bear” is misleading, however. This species can range from black to gray to cinammon to white depending on the location and the individual. To ensure proper identification of an American black bear, do not depend on the bear's coloration.

The NPS also provides a helpful infographic to help distinguish the two: Brown/Grizzly vs. Black Bear

Side note: I learned that grizzly bears and brown bears are the same species. Their only distinguishing feature is their location. Grizzly bears live inland, while brown bears live on the coast, e.g. in Alaska

You can see it's not always trivial to tell the difference. So I decided to build a simple image classifier using fast.ai to help me settle my hiking debates and potentially save my life.

The Data

All good classifiers need a lot of data. Since I'll be using resnet34 and resnet50 as a base to build my classifier, I won't need a ton of new data, but the more data, the better. In this case, I'll need a bunch of images classified as “brown” or “black” to train and test my classifier.

Luckily, ImageNet has a decent collection of images classified accordingly:

Now, I couldn't download the images directly as an independent party, but I did have access to a list of URLs for each image set:

import urllib.request

black_bear_images_link = 'http://image-net.org/api/text/imagenet.synset.geturls?wnid=n02133161'   
black_bear_image_urls = urllib.request.urlopen(black_bear_images_link).read().decode()

brown_bear_images_link = 'http://image-net.org/api/text/imagenet.synset.geturls?wnid=n02132136'   
brown_bear_image_urls = urllib.request.urlopen(brown_bear_images_link).read().decode()

One I did that, I could go through each image and download it to my server. I decided to save all brown bear images under data/bears/brown and all black bear images under data/bears/black.

import pathlib
if not os.path.exists('../data/bears'):
    os.makedirs('../data/bears')
if not os.path.exists('../data/bears/brown'):
    os.makedirs('../data/bears/brown')
if not os.path.exists('../data/bears/black'):
    os.makedirs('../data/bears/black')

path = pathlib.Path('../data/bears')
brown_path = path/'brown'
black_path = path/'black'

Once I had the directory structure established, I could download the images with the following python snippet:

from PIL import Image
IMAGE_NO_LONGER_AVAILABLE = 2051
# download black bear images
for pic_num, i in list(enumerate(black_bear_image_urls.split('\n'))):
    file_name = "../data/bears/black/"+str(pic_num+1)+".jpg"
    try:
        if not os.path.exists(file_name):
            urllib.request.urlretrieve(i, file_name)
	     # Check to make sure the image is valid
            img = Image.open(file_name)
            img.verify()
            assert os.path.getsize(file_name) != IMAGE_NO_LONGER_AVALABLE
            
    except Exception as e:
        print(pic_num+1, e)
        if os.path.exists(file_name):
            os.remove(file_name)
            
# download brown bear images
for pic_num, i in list(enumerate(brown_bear_image_urls.split('\n'))):
    file_name = "../data/bears/brown/"+str(pic_num+1)+".jpg"
    try:
        if not os.path.exists(file_name):
            urllib.request.urlretrieve(i, file_name)
	     # Check to make sure the image is valid
            img = Image.open(file_name)
            img.verify()
            assert os.path.getsize(file_name) != IMAGE_NO_LONGER_AVALABLE
            
    except Exception as e:
        print(pic_num+1, e)
        if os.path.exists(file_name):
            os.remove(file_name)

There are a few things I'd like to note about this script:

  • I include Pillow's Image module to call img.verify(). This raises an exception whenever an image is corrupted or cannot be opened.
  • I also check to make sure the image size makes sense. I fould several “This image no longer exists” images, and they all had the same size: 2.05kb. The assert raises an exception whenever it downloads one of these images.
  • If one of the above exceptions is raised, it prints out the exception and deletes the file (if it exists).

This, to my knowledge, ensures that the ImageNet data set is good quality.

On the server

Following the fast.ai Lesson 1 format:

%reload_ext autoreload
%autoreload 2
%matplotlib inline

from fastai import *
from fastai.vision import *

bs = 64

fn_paths = [brown_path, black_path]

fnames = get_image_files(brown_path) + get_image_files(black_path)

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

np.random.seed(2)
# create the data bunch based on path
data = ImageDataBunch.from_name_func(path, fnames, ds_tfms=get_transforms(), size=224, bs=bs,
        label_func = lambda x: 'brown' if '/brown/' in str(x) else 'black')
data.classes # ['brown', 'black']

data.normalize(imagenet_stats)

After running this, I sanity checked the classifications using data.show_batch(rows=3, figsize=(7,6)). There didn't appear to be anything blatantly wrong.

Training

Again, following the fast.ai Lesson 1 format:

learn = create_cnn(data, models.resnet34, metrics=error_rate)
learn.fit_one_cycle(4)

This yielded the following results:

Total time: 01:03
epoch  train_loss  valid_loss  error_rate
1      0.354983    0.098601    0.036939    (00:19)
2      0.208561    0.131596    0.039578    (00:14)
3      0.156123    0.127603    0.036939    (00:14)
4      0.125690    0.121351    0.036939    (00:14)

That seemed consistent with what I expected: one minute of training and about a 96.5% correct classification.

Trying to

Checking the top confusions:

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_top_losses(9, figsize=(15,11))
interp.most_confused(min_val=2)

From the first plot, I was able to check some of the errors. I happened to notice an interesting classification:

Classified black bear, actually brown

ImageNet had this one classified as a black bear. I checked the watermarked website, and lo and behold:

Actual screenshot from photographer's website

Looks like the dataset had a mis-classified entry! Good thing I checked. It may not have effected the accuracy too much, but it's good to know.

Let's try to fine-tune the results using a learning rate varied over layers:

learn.unfreeze()
learn.fit_one_cycle(2, max_lr=slice(1e-6,1e-4))

Output:

Total time: 00:38
epoch  train_loss  valid_loss  error_rate
1      0.082740    0.113588    0.036939    (00:19)
2      0.079821    0.118543    0.036939    (00:18)

Hmmm, not much change. Let's try training on resnet50 this time.

data = ImageDataBunch.from_name_func(path, fnames, ds_tfms=get_transforms(), size=299, bs=bs//2,
        label_func = lambda x: 'brown' if '/brown/' in str(x) else 'black')
learn = create_cnn(data, models.resnet50, metrics=error_rate)
learn.fit_one_cycle(8)

Output:

Total time: 07:17
epoch  train_loss  valid_loss  error_rate
1      0.220857    0.175432    0.044321    (01:24)
2      0.162850    0.182251    0.052632    (00:49)
3      0.133017    0.143796    0.049862    (00:50)
4      0.095450    0.154293    0.047091    (00:50)
5      0.072514    0.144659    0.041551    (00:50)
6      0.069240    0.112004    0.033241    (00:50)
7      0.042691    0.118620    0.036011    (00:50)
8      0.032191    0.119663    0.036011    (00:50)

And tuning:

learn.unfreeze()
learn.fit_one_cycle(3, max_lr=slice(1e-6,1e-4))

Output:

Total time: 03:26
epoch  train_loss  valid_loss  error_rate
1      0.036145    0.122415    0.024931    (01:15)
2      0.029445    0.118053    0.030471    (01:05)
3      0.025955    0.118540    0.027701    (01:05)

Looks like we improved our error rate by about 1% over resnet34. interp.most_confused(min_val=2) now yields ten mistakes in the whole dataset (400+ images).

Reflections

Overall, this was a pretty solid learning experience. I tried to solve an interesting problem, and it worked pretty well.

My most learning came from the dataset creation portion. I had initially assumed that the ImageNet data was actually in reasonable shape. It turns out only about 15% of the images were still available and non-corrupt! I'll try downloading directly from Google or Bing next time to see if I can get higher-quality data for my next claissifier.

I didn't have to change much of fast.ai's boilerplate learning code to achieve my 97.5% accuracy. Of course, I haven't actually tested this on outside data yet, so I'm making a big assumption that the results hold. If they do, it really shows the power of training for specific classification on a pre-trained, general image model!

...of a landscape, specifically.

Intended Audience

Amateur photographers who want to start taking landscape photography seriously. You are mostly familiar with how to do things on your camera like change the exposure compensation or white balance profile, and can figure out these details if needed.

Skip this post if...

  • You don't care about taking nice landscape photos
  • Your camera of choice isn't a DSLR or mirrorless camera. Not having one of these is totally fine, and oftentimes the best camera for a photo is the one you have with you. But I can only speak from experience from using DSLR and mirrorless cameras — Nikon D3300 and Sony Alpha a6000, respectively.

Okay, with that said, let's get into the guide!

The tools

1. Your camera – I think most of you will have this one.

2. A good lens – The starter lens on most cameras — 18-55mm for Nikon, 16-50mm for Sony, or the equivalent for Canon — will do. No need for fancy lenses yet since we're just learning.

3. A tripod – since we'll be shooting most of our photos with a high aperture, the camera's shutter (or equivalent) will be open for a while, relatively speaking. Image stability requires a tripod.

4. A remote shutter release – pressing the main shutter release button on the camera can jostle the camera, so to avoid this, use a remote shutter release for your camera's model. I use a wired shutter release, but I've seen wireless versions be effective too.

5. A filter, depending on what you're shooting. For waterfalls, rivers, or general landscapes during the day, you will need a good polarizing filter. For sunsets and sunrises, you will need some graduated split neutral density filters. I've linked mine, but you don't have to drop that much cash for a nice photo.

The steps

Let's say you've found a nice spot to take a photo. How do you go about doing it?

1. Get set up Put your camera on your tripod, your filter on your camera, and the shutter release in your camera. Make sure your camera is in aperture priority mode (the dial at the top is pointing at the “A”). This should take around 30 seconds, and if it takes longer, practice this process at home when you have a spare minute.

2. Compose your image Position the camera so you have a nice composition. This is more art than science, but some general guidelines I use are the rule of thirds and to create visual paths between parts of the image with its components. This second point keeps the viewer's eyes on the image for a longer period of time. As an exercise, try to look for this concept the next time you're looking at photos.

3. Adjust your focus Most of my landscape photos use f/22 aperture. Waterfalls generally vary from f/13 to f/22. Because your aperture is so small, you can focus on foreground components and the background will, for the most part, also be in focus. I use the automatic focus to get close to what I want, then I flick the little switch on the lens from “A” to “M” and fine tune the focus with the focus ring until I have the foreground object I want in focus.

4. Adjust your filters With a polarizing filter, try to make the image as dark as possible. With graduated split neutral density filters, try to find a clean break between elements (i.e. the horizon) and position the edge of the dark portion there.

5. Take your picture For the trained photographer, the above steps should take around 60 seconds. Now that we have everything set up, we can take our pictures. A few things to try doing here:

  • Adjust the exposure compensation. I generally find that -0.33 gives me the best results, but your mileage may vary.
  • Adjust the white balance. Your camera has several pre-programmed white balance settings: “cloudy”, “direct “sunlight”, and “shade” are the ones I use most often. Use the one that looks best for this photo.
  • Bonus: you can adjust the sharpness, contrast, and saturation of the photos you take in the Picture Control settings. How to do this is camera dependent, so I'd look up “<camera make and model> adjust picture controls” on Google.

That's it! Following these steps, or at least keeping the composition step in mind, should help you take better landscape photos. Also keep in mind that many of the landscape photos you see online are edited in Photoshop/Lightroom, and that a “perfect photo” is often the result of patience, good timing, and luck in addition to good composition. That said, the only way to learn the “good timing” and “good composition” components is to practice, so get out there and capture that pretty scenery!

I've finally done it. I've deleted my Facebook account.

I don't expect any kudos for this news; deleting your Facebook account is the prudent thing to do these days. Heck, my mother deleted her account seven months ago. For the last several months, my account has been deactivated, my profile a blank slate: no statuses, no photos, no likes or shares. The only reason it survived this long was Messenger, since I did not want my friends to have a hard time reaching me.

So what changed? What finally caused me to press the Big Blue Button?

I have an unlimited mobile Hotspot (from the Calyx Institute) so I can access the internet wherever I have a reliable signal, which is in most urban areas. Oftentimes, when going about my day, I quickly disconnect and reconnect, which occasionally assigns me a new IP address from a nearby city. Some sites, including Facebook, interpret this as a “new location.” Trying to log in from this “new location” locks your account, as the site thinks you're a bad actor.

Now, other sites like Discord handle logging in from a new location reasonably, in my opinion: they make you complete a captcha, send you an email, and — boom! — you're back in. A little annoying, sure, but not too tedious. Facebook does not take this approach. Instead, it asks you to input personal information about yourself and others, which I am unwilling to do.

I was presented with three options to verify my identity and unlock my account:

  1. Enter your phone number. I did not have my phone number attached to my Facebook account, so I am unsure how they can verify this information. Maybe a friend of mine imported their contacts into Facebook and my name and number was on that list. Oh well, not much I can do about that. But I'm not going to validate my number for them, so what's next?
  2. Tag your friends in photos. This one seems innocent enough, but I feel like tagging my friends in photos is helping Facebook train its facial recognition software, and I don't know if my friends consented to that. I feel like selecting this option would be violating their right to privacy, so it's also a no-go. What's left?
  3. Verify your birthday. Ah, yes, the only piece of profile information that remains on my profile. I guess this one is okay since it's on my profile. I entered my birthday, but I decided that I didn't want Facebook to know it from now on. Thus, once I logged back in I tried to remove it from my profile. Turns out you can't. I get it, they have to make sure I'm over 13, but nobody lies about their age on the internet, right? ¯\_(ツ)_/¯

I managed to avoid being locked out this time, but what if I just got lucky? What if next time it happens they ask me for government-issued ID or something similar?

That realization convinced me. I logged on to Messenger, let the few close friends who didn't have my phone number know about my impending account deletion and how to contact me, downloaded Facebook's data on me — 2 GB of it, even though I have no interests, no posts, no photos, and no videos...must be all my messages! — and deleted my account.

Goodbye, Facebook. I was willing to give you a chance, but your hunger for my personal data forced my hand. I only hope that it was soon enough to regain some semblance of privacy in my life. I have a sinking feeling, however, that I gave that up when I signed up for an account in 2009.

Do not fail.

This goal overrode all others during my teenage years. It drove me to a high-achieving high school career during which I maintained a perfect GPA on an AP course load. The fear of failure kept me up at night; every assignment was a high-stakes affair.

When I did fail, I tried to cover it up or lie and pretend my record was blemish-free. When I scored a 4 on a single AP test (out of six; the rest were 5s) and a friend asked me how I did, I took a picture of the sheet, copied a “5” from elsewhere on the page and pasted it over the “4”. Looking back now I can see how ridiculous that was, but I had a genuine fear of letting weakness show, especially when it came to grades. If I let weakness show, I would consider myself a failure.

If I could not cover up my failure, I resigned myself to it and stopped trying to improve. I often resorted to this tactic during my competitive running career. Once I realized I would not win my races with God-given physical gifts alone — I was shorter and stouter than many high school runners — I said to myself, “I'll just run on the high school team to stay in shape.” That kept me on the fringes of varsity, but I will never know how I would have done if I actually tried to improve.

I share these details because I know that others can relate to my fear of failure and belief in “natural abilities”. This belief that skills and intelligence are fixed and cannot be improved through hard work and perseverance is called the Fixed Mindset. It is a pervasive mindset even among world-class performers.

In retrospect, I realize I had fixed mindset for years, even after I joined Microsoft out of college. Satya Nadella, Microsoft's CEO, encourages all employees to embrace the Growth Mindset, or the idea that any ability or skill, even general intelligence, can grow (i.e. improve with work and perseverance). My initial, unironic response to his request: “Oh, growth mindset isn't for me...I'm too stuck in the fixed mindset.” I wish I could say that was satire.

Some time after I left Microsoft, I picked up Satya's biography Hit Refresh. In it, he talks about how he learned about the growth mindset through a book his wife gave him called Mindset. Then, shortly after I read this passage in Hit Refresh, a friend recommended I read Mindset. Coincidence? Maybe, but that was enough for me. I bought the book that evening and read it over the next few weeks.

Mindset, by Dr. Carol Dweck, discusses both the fixed mindset and growth mindset, as well as the research behind it. It made me realize how I developed my fixed mindset, how to avoid transmitting the fixed mindset “disease” to my students and younger cousins, and how to “cure” myself — in other words, embrace the growth mindset. I plan to try it in December, then into 2019, as well as work on my metacognition skills so I can catch myself if I slip back into the fixed mindset.

My main takeaways from my reading and reflection: – Fixed mindset is common. I have it, and I'm sure some of you reading have it as well. Many people in my life have it, which may be why I developed it in the first place. – “Failure” is a valuable learning experience. One case study claims it is even more valuable than success. I would say that critical reflection on any attempt, success or failure, is essential to development of a skill. – Fixed mindset is fixable. You don't have to be stuck in the fixed mindset forever. You can improve in areas you previously thought “weren't for you.” However... – Growth mindset takes time to develop. It will not happen overnight, and sometimes it will be a struggle. Perseverance to the book's teachings, examination and learning from your failures, and belief in the book's teachings will see you through these speed bumps.

I hope that by this time next year, I can say I've fully embraced the growth mindset. For more details about fixed mindset, growth mindset, and how to use growth mindset in your own life, I would highly recommend reading the book on the subject and joining me on my journey!

I love to hike.

For those of you that know me, this comes as no surprise. I go hiking almost every weekend, and it usually winds up in our conversations somehow. For those of you that do not, I will probably mention it at least a few (dozen) times when we meet. Hiking is my main hobby and has been for as long as I can remember.

Sure, hiking itself is not very interesting. All you do is put on some shoes, throw some snacks, water and supplies into a bag, drive for a little bit — if at all — into nature, and start walking. Why have I stuck with it for more than two decades?

The simple answer: I have way too much fun while hiking. It is also the open secret to my good health and general well-being. And if you have not been hiking recently, you should give it a shot. Let me explain why.

The most obvious reason is the physical benefit. Recreational hiking is not the same as a regular workout routine for getting into good shape, but some physical activity is better than none at all. If you are not used to climbing mountains or walking for miles, start with a walk down the street. Once you can do that, go to your local park and hike the shortest trail. Work up from there. Keep at it; your doctor will thank me later.

Your mental health will also benefit from hiking. Aside from the well-documented dopamine rush from physical activity, being out in nature has shown to improve mood. Even if it's just a walk in the park during your lunch break at work, you should come back feeling refreshed and maybe even a little energized. One important caveat here: turn off your cell phone. Separate yourself from your work, and stop thinking about it. Focus on your surroundings or let your mind wander!

That brings me to the professional benefits to hiking. This section is especially important to STEAM workers — that is, creatives, engineers, researchers, etc. Chances are good that your job involves challenging and difficult problems. It follows that creative insight can greatly increase your odds of solving them, thereby advancing your career. As it turns out, one of the proven ways to solve a problem is by not thinking about it for a while. In Rest: Why You Get More Done When You Work Less, Alex Soojung-Kim Pang explores the process of generating creative insights. One section, called Walking, describes the benefits of a daily walk on creativity. Most of the evidence provided is testimonial, but studies did show that walking correlates with an increase in divergent thinking, a.k.a. creative thinking. Other sections explore the benefits of “deep play” — recreational yet physically challenging activities that require complete focus — and exposure to nature for creativity. Honestly, the book provides a better argument for regular hiking than I can in this short article. I would recommend reading it when you get the chance.

There are surely other reasons than these to hike. There are probably other ways to achieve its benefits. I find, however, that regular hiking relieves my stress, energizes me, makes me happy, and improves my physical health. It is the second biggest reason for my well-being after a good night's sleep. If you made it this far, please consider taking up my “hobby”, and I hope to see you on the trail!