Timo Denk's Blog

Bay Area Commute Time Tracking

· Timo Denk

Google has several campuses scattered across the Bay Area, and I typically work from either San Francisco, San Bruno (YouTube), or Mountain View. Getting to the latter is (at best) a 32-minute drive, though that strongly depends on the day of the week and the departure time.

While I enjoy listening to audiobooks or just letting my thoughts wander while driving, it’s still clearly preferable to not be stuck in traffic. In order to determine the optimal departure times, I called the Google Maps API more than 50k times over the course of five months (mid-January through mid-June 2024) to collect some data.

Maps commute screenshot The collected data is what you’d obtain if you visited Google Maps every couple of minutes and read off the estimated time en route.

Results

By crunching the data, we obtain this plot, which shows the median time it takes to commute from 250 King St in San Francisco to Gradient Canopy in Mountain View (and vice versa) on each day of the week. The fine lines are the individual measurements with some smoothing applied.

Commute time plot SF to MTV

Some observations are:

  • Weekends are mostly chill.
  • To avoid the morning traffic, I have to leave either before 6:40am or after 10am.
  • On the way home, the cutoff is at about 7pm (that’s easy :P).
  • Mondays and Fridays are less intense than the other workdays.
  • Afternoon traffic picks up surprisingly early at 2pm and peaks around 5pm.
  • Between 10pm and 4am it is smooth sailing on any given day.

Technical Setup

I’ll briefly share the setup used to collect the data. The project is on Google Cloud. It has the following architecture:

Commute time tracker system architecture

A Cloud Scheduler Job regularly calls a Cloud Function. It has a single source code file (see the next section of this article) which has a function for calling the Google Maps API. The results are then written into a Cloud Storage Bucket. Using a separate script (in a Colab), I can retrieve the data from the bucket and create the plots.

Although this setup may be a little overengineered for the task at hand, it ran without any issues for the entire five months. Using the Cloud Scheduler was giving me some nice flexibility in timing the tasks. The page there shows a table like this:

NameFrequencyTargetLast run
home-to-work-high-freq*/3 7-19 * *1-5 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 15, 2024, 4:57:00 AM
home-to-work-low-freq-1*/5 0-6,20-23 * *1-5 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 15, 2024, 8:55:00 AM
home-to-work-low-freq-2*/5 * ** 6,0 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 16, 2024, 4:20:00 PM
work-to-home-high-freq*/3 7-19 * *1-5 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 15, 2024, 4:57:00 AM
work-to-home-low-freq-1*/5 0-6,20-23 * *1-5 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 15, 2024, 8:55:00 AM
work-to-home-low-freq-2*/5 * ** 6,0 (America/Los_Angeles)URL : …/maps-api-caller-publicJun 16, 2024, 4:20:00 PM

It allowed me to conveniently reduce the sample rate at night and off rush hours to obtain high resolution only during workdays between 7am and 7pm. The costs were minimal because there is a monthly credit of $200 for the Maps API which I used up exactly every month.

Overall, this was fun to work on, and it’s the second time I’ve done this sort of analysis. I now have these plots readily available and can refer to them when deciding whether to play the violin for another 30 minutes before making the drive down. :-)

Source Code

Below is the JavaScript code running on the Cloud Function.

const functions = require('@google-cloud/functions-framework');
const { Storage } = require('@google-cloud/storage');
const https = require('https');
const { parse } = require('csv-parse/sync');
const { stringify } = require('csv-stringify/sync');

const storage = new Storage();
const BUCKET_NAME = 'maps-commute-data';

const MAPS_API_KEY = '<insert>';
const MODE = 'driving';
const TRAFFIC_MODEL = "best_guess";

function getRequestURL(key, origin, destination, mode, trafficModel) {
	return "https://maps.googleapis.com/maps/api/directions/json" + 
		"?origin=" + encodeURI(origin) + 
		"&destination=" + encodeURI(destination) + 
		"&mode=" + encodeURI(mode) + 
		"&trafficModel=" + encodeURI(trafficModel) + 
		"&departure_time=now" + 
		"&key=" + key;
}

function getShortestDuration(responseBody) {
  console.info(responseBody);
	if (responseBody.routes.length == 0) return NaN;
	let shortest = responseBody.routes[0].legs[0].duration_in_traffic.value;
	for (let i = 1; i < responseBody.routes.length; i++) {
		shortest = Math.min(shortest, responseBody.routes[i].legs[0].duration_in_traffic.value);
	}
	return shortest;
}

function appendLogToCSV(fileName, duration, callback) {
  const bucket = storage.bucket(BUCKET_NAME);
  const file = bucket.file(fileName);
  
  file.download((err, data) => {
    let csvContent;
    if (err) {
      if (err.code === 404) {
        csvContent = [];
      } else {
        return callback(err);
      }
    } else {
      csvContent = parse(data.toString(), {
        columns: true,
        skip_empty_lines: true
      });
    }

    csvContent.push({ timestamp: new Date().toISOString(), duration: duration });
    const updatedCSV = stringify(csvContent, { header: true });
    file.save(updatedCSV, callback);
  });
}

function formatForFileName(str) {
    str = str.toLowerCase();
    str = str.replace(/[\s]/g, '_');
    str = str.replace(/[^a-z0-9_]/g, '');
    if (str.length > 40) {
        str = str.substring(0, 30) + str.substring(str.length - 10, str.length);
    }
    return str;
}

functions.http('callMapsAndLog', (cloudFnReq, cloudFnRes) => {
  const origin = cloudFnReq.headers.origin;
  const destination = cloudFnReq.headers.destination;

  const logfileName = `log-origin-${formatForFileName(origin)}-to-${formatForFileName(destination)}.csv`

  let url = getRequestURL(MAPS_API_KEY, origin, destination, MODE, TRAFFIC_MODEL);
  https.get(url, (res) => {
    let data = "";
    res.on('data', (d) => { data += d; });

	res.on('end', () => {
		let duration = getShortestDuration(JSON.parse(data));
		appendLogToCSV(logfileName, duration, (err) => {
		if (err) {
			cloudFnRes.status(500).send('Error writing to CSV: ' + err.message);
		} else {
			cloudFnRes.send(`Shortest duration from ${origin} to ${destination} is ${duration}s.`);
		}
		});
	});
  }).on('error', (e) => {
    cloudFnRes.send(e);
  });
});

With the following package.json

{
  "dependencies": {
    "@google-cloud/functions-framework": "^3.0.0",
    "@google-cloud/storage": "^7.7.0",
    "csv-parse": "^5.5.3",
    "csv-stringify": "^6.4.5"
  }
}