Indexing Google's Source Code

I couldn't talk about this before, but now that Wired magazine has disclosed the size of Google's code base (2 billion lines), I can discuss my authorship of gtags and what it did for Google, as well as some funny stories arising from that.

I wrote the first version of gtags in 1991 (yes, gtags is older than Google!), when I was at Geoworks. GEOS was several million lines of assembly, including every freaking app written for that OS. Since every object could potentially call any other object, the entire code base was relevant. Needing to get my head around that code base, I tried to build a TAGS database and that immediately caused my Emacs to start swapping. The performance was unacceptable.

The core insight was this: there's no reason to use a disk based search on a TAGS database. Stick the entire database into RAM, and use a hash-table to lookup your keywords, and search performance would go from multiple seconds (half a minute in some cases) to sub-second response time. So one weekend I coded up the hash-table, wrote code to load up a TAGS database into memory, and implemented a simple UI that let me browse code in Emacs. Soon, I enjoyed sub-second search times and could grok code that would have been impossible to do any other way.

If I ever needed validation that the tool-building approach to coping with large-scale software was the right approach, this was it. Once the senior engineers (remember, I was an intern at Geoworks then) got hold of the tool, I saw even loyal-vi users switch over to Emacs just to get their hands on the code browsing functionality (going from half a minute per search to sub-seconds was critical).

After I left Geoworks, most of my code was in C, C++, or other high level languages. Computers got so fast, and IDEs so sophisticated that I never dealt with a code base that couldn't be loaded into the IDE. It seemed to me that the need for such functionality had been obviated by ever more powerful machines.

That was, until I joined Google in 2003. By then, Google's code based was already approaching 1 billion lines, but in multiple languages. I needed to wrap my head around that code base in a hurry. Various teams were using random tricks to subset Google's code base into their IDEs, which I thought was a kludgy and unsatisfactory way to work. So in my 20% time, I rewrote my old tool using Google infrastructure (thanks to Craig Silverstein, who was the only person who believed in my code enough to waste precious time code reviewing it --- even then he was skeptical that my tool would be widely used or even substantially useful, given the huge amount of effort people had put into subsetting the codebase). I coded up the UI again in Emacs Lisp. I actually had to put some effort into the UI this time, given that C++ (and Java) overloading meant you had multiple search results for any given search term. Thankfully, Arthur Gleckler came in to lend a hand. Reading Arthur's Lisp code was like reading poetry: you can't believe the succinctness and elegance that can be expressed in so little space. It's worth your time to learn Emacs Lisp just so you can read Arthur's code.

Just as I expected, gtags took off in a huge way inside Google's engineering team. (By the time I left, 2500 daily active users was the metric, or about 25% of Google's engineering workforce. The internal tools team did a survey once and discovered that nearly every engineering workstation had a copy of Stephen Chen's gtags-mixer running on it) There wasn't a whole scale conversion from vi to Emacs though: Laurence Gonsalves stepped in and wrote a vim script that emulated the Emacs code. I don't even remember how I managed to do the code review for that checkin, but anything to help gtags, so I must have just gritted my teeth and done the code review or asked Laurence to find someone competent to review it.

But I wasn't nearly even close to done. Because of the huge amount of ambiguity and overloading involved in C++ and Java, I wanted gtags to do a better job of ranking the results of any given search. Phil Sung took a first crack at it, introducing Sung-ranking and later on, an include-rank that mirrored page-rank, except for code. Stephen Chen solved the problem of how to intermix protocol buffer files into the search results. Matei Zaharia (now a professor at MIT) spent a summer integrating a parser into the indexer for gtags, so that it was no longer a dumb lexical scanner but a full-on type-comprehension system for both C++ and Java. He also designed and implemented incremental indexing on Google's code base, no mean feat. Leandro Groisman and Nigel D'Souza both also made major contributions to gtags.

For several years, I had the entire Google source repository downloaded and checked out on a dedicated gtags indexing machine sitting under my desk. It was a standard underpowered workstation of that era: dual core, 2GB of RAM, and 500GB of disk: it had a special p4 client that eliminated the need to download any binary assets, since it was only interested in code! It was probably a major security hole, but I figured since Bill Coughran knew about it, I wasn't violating any corporate policies.

This illustrates a very important point: 2 billion lines of code sounds like a lot of code, but if you do the math (assuming 50 characters per line) you'll get only about 100GB of data (uncompressed). After dropping comments, white space, and lines that don't perform any declarations, your index is going to be pretty tiny, and you need to split that code base into several corpora (C++, Java, protocol buffer declarations, python), so each individual server could easily handle its entire corpus in RAM without any fancy sharding. Too many people get caught up in trying to apply fancy Google techniques required to manage terabytes of data when they're dealing with tiny amounts of data that fit into RAM and can be managed by traditional programming techniques.

In any case, gtags was a very hardware light project: it never took more than one machine to index all of Google's code base (and we never had to apply any fancy MapReduce techniques), nor did the serving cluster ever exceed more than about 10 machines. We came close to maxing out the RAM available on 32-bit machines for a while, but between Phil's string table optimization reducing memory use by 75% and the switch to a 64-bit architecture we never ever had to split indexes for any given language (there was a server for each language) across multiple servers. Those servers were under-utilized of course (they could probably have served 25,000 or 250,000 users at once), but on the flip side, you always got sub 10ms response times out of gtags. We switched from dedicated gtags server desktops sitting under people's desks to Google's cloud internally fairly early on, with Ken Ashcraft doing much of the work of converting gtags into a borg-ready service.

This came to a head when Google added the China office sometime in 2005 or so. After that, the powers that be decided that high intellectual property (HIP) code needed special permissions to access. Since I wasn't HIP enough, I simply stopped indexing that code. This burdened the HIP people so much that eventually some of them (including Sandor Dornbush) contributed to gtags. A HIP guy would take on the burden of downloading HIP code and indexing it using our indexer, and then put up the gtags server with that code behind a HIP firewall. The gtags-mixer would then be configured to talk to the HIP server and mix-in the result if you were HIP enough.

One of my prouder moments at Google was when Rob "Commander" Pike came to me and asked me how gtags worked. It turned out that he didn't want to talk to the gtags mixer or the gtags server, but just wanted his programming environment/editor to directly grok the output of the indexer. I was happy to give him access to the index for him to do whatever he wanted with it. I forget the mechanism by which this happened: he might have simply scp'd the index over to his machine, or I might have had the indexer push the index to his machine whenever it was done. This was great, because Rob became one of the folks who would notice whenever the indexer was broken because the file wouldn't get updated!

In any case, as with many things at Google, after I left gtags got replaced by some cloud solution that took way more resources than me, Arthur, and a bunch of interns, and I'm sure everything I wrote has been long retired by now, with the possible exception of the Emacs Lisp front-end.

Even after I left Google, gtags paid me back. Soon after I met my wife, she talked to some of her friends at Google about who she was dating. One of them did a p4 lookup on my changes, and said to her, "Hey wow, this guy has code commited everywhere, even the protocol-compiler." So I guess that worked out as far as a back-door reference check was concerned. (That change in the protocol-compiler was necessitated because I wanted to inject a clue in its output: that clue enabled the gtags indexer to map a generated C++ source file back to its original .proto form --- it was far easier to do that by having the protocol compile emit the clue than to try to guess --- it was a trivial change and Sanjay approved it in seconds)

If it seemed unbelievable to you that during that period of time I had such an illustrious group of people on a tiny 20% project, it should be. But I maintain that the test of a high quality engineering organization is whether or not that organization is able and willing to invest time, money, and effort into building tools that enable that organization to move faster and produce higher quality code. Google met that test and passed with flying colors.

Review: The Opposite of Spoiled

I checked out The Opposite of Spoiled hoping for practical tips on how to teach children about money. I got more and less than what I bargained for.

In particular, the people this book are written for aren't anywhere close to my profile. These parents are white, privileged, have ridiculous amounts of money, and don't spend time with their kids. If you're Asian, come from an unprivileged background, and don't take private jets to fly to your vacations, you're probably in no danger of needing to read this book whatsoever. To be fair, I've met a large number of people who meet the above category, so this is not to say that the book is useless. It's probably very useful for white rich people in the 1%. It just makes no sense to read or follow its prescription if you're not.

Here are a few examples: the book describes a common problem of over-parenting as parents waiting in line outside summer camps rushing to their kids with all sorts of goodies, seeking to outdo each other. These summer camps seem cushy compared to the kind of trips I've already taken my 3 year old on. I can't imagine that I'd ever worry that my son couldn't make it a few days without his iPad. He's made weeks without any kind of electronic device. I imagine parents who have no idea what backcountry camping is would have the kinds of challenges described in this book.

Another example: the book presumes that you want to teach generosity to your kids. If you come from an immigrant background, especially if you were brought up in Asia, you learned to value society over the individual. Your problem isn't teaching your kids how to be generous and nice, Your problem is teaching kids how to be sufficiently selfish that they wouldn't get stepped on in the extremely competitive (by comparison) American society. I remember being horrified by story after story ex-Googlers told me about political behavior from my peers all in the name of getting ahead. Most of those stories involved people of privileged stepping on H1B holders or taking advantage of people who valued societal niceties over every man for himself. (I myself have had people tell me that it was my biggest barrier to succeeding)

What are my recommendations? If you're a white person, please read this book and apply it to your kids. If you're Asian, you'd do far better by pretending this book doesn't exist. And don't worry about teaching finance to your kids. Just by being Asian your example would probably be sufficient.

Teton/Yellowstone RV Trip: Tips, Conclusions, and Brief Equipment Reviews

When I planned the trip, the thought was that Bowen would love it, and I'd be indifferent. The reality was that Bowen loved it, but I constantly chafed at driving a giant-ass RV around. The fact is, there were too many roads in the National Parks that aren't accessible if you're driving an RV. If I were to do it again, I'd take the advise that the owner of Utah RV Rental gave to me as he was driving us to the airport: rent a towing vehicle and a trailer. That way you can dump the trailer in the campground and drive around in the towing vehicle and still access all the nice roads. Apparently, it's even cheaper! The reason he didn't recommend it to first-time RV renters was that it makes backing the vehicle much harder.

But beyond that, the biggest problem is that the Tetons and Yellowstone are just way too crowded in the summer. I don't enjoy spending time in traffic jams to begin with, but having to do so on vacation makes things even worse. I've always visited those areas in September, and never realized how much worse it was in the summer. And the reality, of course, is that my many summers in the Alps have spoiled me: neither the Tetons nor Yellowstone look pretty enough to me to justify the effort, and I'd take a week in the Alps over 2 weeks in those parks any day. (As far as I'm concerned, Glacier National Park is pretty much the only park in the US where the natural beauty comes close to what you can get in the Alps)

Finally, getting a good experience RV camping is much the same as getting a good experience in any outdoor endeavor: let the weather drive where you go and what you do, and don't let any plans disrupt that principle. In the summer, your early morning hours are by far the most valuable hours in your day, and picking what you do then strongly determines how smoothly the rest of the days go. If that means you wake up early, do it!

Several pieces of equipment were standout useful during the trip:

• External Battery Packs: The RAVPower 10400mAh paid for itself multiple times, keeping my phone charged while driving the extremely long distances. The 3200mAh bank was also useful because it incorporated a flashlight, which was useful for other members of the party.
• Garmin Vivoactive: Look for a long term review to come in the future. But let's just say that I'm very satisfied with this device, and it's really as good as it gets.
• Bestek Inverter: You'll typically drive a lot more than you'll run the generator on this trip, and this handy and inexpensive device was what allowed me to keep my CPAP battery charged and ready for those nights without power. A literal life-saver.
• Nokia HERE: Google still doesn't acknowledge that there's a world in which you won't have internet access everywhere. Well, Nokia HERE allows offline navigation, search, and routing. It's also a ton less power hungry than Google Maps. The download interface for maps is well done, and reliable. This is the must-have app if you're going to visit National Parks or going outside cell signal range.
• Retevis Walkie-Talkie: Since cell signal doesn't work, if you have a big party, you want to be able to use walkie-talkies to coordinate when people wander around. At $55 for 3, these proved reliable and useful. Well worth the money.
• Sony SBH52: If you have to manage a toddler while potentially having to make or receive phone calls, a bluetooth headset is a must. This one is hard to beat because it's water resistant (even the charging port is sealed), is extremely loud, and even better, when phone calls need to be made, you can remove the ear-buds and hand it to other members of the party to treat as a normal phone. I spent many hours driving with the earbuds in my ears so I could hear the directions from Nokia HERE in my ear while the kids and adults in the RV were talking at the top of their voices.

I doubt if I'll do another trip like this any time soon, but if you're going to attempt one, I hope these tips help.

Teton/Yellowstone RV Trip Day 15: Ogden to Salt Lake City

We got up early, drained the RV of all the essentials, and drove it back to Utah RV Rentals, where the checkin process revealed that we'd driven 300 miles more than our free allowance, used up 3.5 gallons of propane, and only used about 4 hours of generator. We got our free ride to the airport, but since our flight didn't depart till 9:50pm, and Delta wanted $50/person to fly standby on an earlier flight, we elected to spend the money by renting an SUV instead and driving up to Park City to have lunch, dinner, and enjoy the mountains.

Park City had these fun ski-resort things to do in the summer, like the Alpine Coaster, the Alpine Slide (which Bowen loved because he could do the controls, and so he did it twice, once with me and once with Grandma).

It's a nice town by American standards, but of course nowhere as picturesque or built with good taste as the equivalent European ski towns I've visited in the past summers.

The flight was easy and non-descript, and we were happy to be sleeping at home instead of a cramped RV.

Teton/Yellowstone RV Trip Day 14: Lava Hot Springs to Ogden

We went tubing in the morning, with Bowen hobbling his way to the rental shop, and then hobbling to the tubing put-in. I was very impressed that the little guy lived up to his word. I was actually a little concerned about the tubing portion, because the tube was constructed in such a way that he could only hang on to one handle at a time. Fortunately, the river was shallow and I could maneuver the tube using my feet. Keen sandals have proven their worth to me over and over again, and they did not fail me this time.

I eventually figured to have him sitting up, while I held on to his life jacket over the whitewater sections. It was by no means an easy balancing act, but it was safe under those very mild conditions. With his low walking speed, we could only manage one run in the hour we had, but he didn't protest (probably because he was tired from all the walking).

We drove the vehicle down to Ogden, where we found an RV site with full hookup and excellent facilities, including a swimming pool. We ate out for lunch to celebrate the end of the trip and the last night of camping, and started the job of packing everything.

In the afternoon, I took Bowen to the swimming pool, and spent a couple of hours there because it was so hot. We made friends with a 10-year-old girl who started talking to us by saying, "Are you Chinese? Because I don't like Chinese people." These little moments make me understand why some people willingly work with children: there's a directness and open-mindedness there that disappears when humans become fully socialized, ironically.

When all was packed, we got ready for the end of our adventure and returning our RV the next day.

Trip Index: Grand Tetons, Yellowstone, and Sawtooth Mountains RV Trip

In August, Bowen, my parents, and two of Xiaoqin's relatives and I took an RV trip from Salt Lake City up through Grand Tetons and Yellowstone National Parks, and then back down to Salt Lake City via the Sawtooth Mountains. It was 1600 miles of driving, and many days of camping, swimming, and hiking in the area.

Because Google Photos can use up all your e-mail quota (that's right, some idiot at Google thought it was a good idea to commingle quota from a critical service with quota from a 'nice-to-have' service that has no way for you to manage quotas), I've switched away from using Google photos or Picasa to using OneDrive for photo sharing. OneDrive is slow compared to Google photos, but at least I won't accidentally use up critical quota, and it comes with decent quota management tools.

This page also serves at the central index for my day-by-day trip journal.

• August 3rd: Salt Lake City to Bear Lake State Park
• August 4th: Bear Lake State Park to Fireside Resort
• August 5th: Rafting on the Snake River
• August 6th: Fireside Resort to Flagg Ranch Headwaters Campground
• August 7th: Jenny Lake
• August 8th: Flagg Ranch to Colter Bay
• August 9th: Colter Bay to Bridge Bay
• August 10th: Yellowstone's Grand Canyon
• August 11th: Lake Yellowstone, Tower Falls and Boiling River
• August 12th: Bridge Bay to Bozeman Hot Springs KOA
• August 13th: Bozeman KOA to North Fork Village
• August 14th: North Fork Village to Wood River National Forest Campground
• August 15th: Wood River to Lava Hot Springs KOA
• August 16th: Lava Hot Springs to Ogden
• August 17th: Ogden to Salt Lake City
• Conclusions, Tips, and Brief Equipment Reviews

Teton/Yellowstone RV Trip Day 13: Wood River to Lava Hot Springs KOA

We woke up to clear skies and drove down to Sun Valley where gas was cheaper (though not cheap) and added gas and bought a few more groceries.

Our destination for the day was Craters of the Moon National Monument.Doug had told me that it was at most half a day, but the reality is that there was a lot to see if you wanted to see every sight. The weather was fortunately a lot cooler than it had been in the past, and we luckily happened upon a ranger guided hike to one of the lava caves. As a result, not only did we get great explication, he also told us that the Indian Tunnel could be traversed safely and we could go through the cave and pop out on the other side like a ground squirrel. So that's what we did. It was  a lot of fun but it consumed quite a bit of time, so we had to move after that.

Our destination for the day was Lava Hot Springs, which not only had Hot Springs but also a swimming pool with water slides, etc. We found a campground with full service but the only site available was a back-in site with lots of obstacles which required delicate maneuvering requiring no less than 3 spotters out of the vehicle providing directions. It wasn't that hard and we were never in danger but it required a lot of patience, fortunately something I'd learned to have when parking sailboats.

After dinner, Bowen wanted to go to the Hot Springs, so there we went. But after half an hour he stubbed his toe on something and started bleeding, so I had to take him back to the RV. He wanted to do the tubing on the river the next morning, but since I'd have to carry the huge inner tube, I couldn't carry him at the same time, so I told him that he had to be able to walk the next morning or we couldn't do any tubing. He nodded and agreed to go to sleep early so we could go tubing.

Teton/Yellowstone RV Trip Day 12: North Fork Village to Wood River National Forest Campground

We left North Fork Village at a leisurely pace, heading up towards Stanley, which was strongly recommended by others. Near Stanley, we asked about where to go, and a local river guide told us to head into Red Fish Lake. Upon entry into the area, we noted that all the campgrounds were listed as being "full."

Despite the free parking area being extremely over-crowded and noisy, we found a paid day use parking site overlooking a swimming area that was serene by comparison. We paid the $6 fee and then went swimming. After the swim we did a short hike along the lake but discovered (not unexpectedly) that the trail was nowhere as well-maintained as those found in National Parks. It being 4:00pm anyway, we headed out of the area to search for a night to stay.

We visited several more national park campgrounds, but all were either full or didn't take RVs. National Forest campgrounds aren't monitored very well, so even if the sign outside the campground (we tried the one at Alturas Lake) says "open", you could easily drive in and discover that it's full. We weren't even close to being tempted by the campground at Smiley Creek.

We headed over the pass out of the Sawtooth Mountain area. The turnouts offered beautiful views, and we stopped several times for photos.

On the descent, we found Wood River campground which was open for RVs. We drove in and found only one spot open, site #20. We parked and I walked over to the campground host to pay, but once there discovered a beautifully shaded site next to her was available. Being no dummy, I quickly paid her for site #3, and then ran back to drive the RV over before anyone got too settled.

Wood River was a beautiful campground, including a small hiking trail that was quite pretty and easy. We enjoyed our stay despite the lack of power, as we managed to hookup to a campground hose and got unlimited water.

Tetons/Yellowstone RV Trip Day 11: Bozeman KOA to North Fork Village

The Bozeman KOA provides breakfast as part of the package, so we got up in time to eat the all-you-can-eat pancake breakfast. It was reasonable, and even the coffee didn't taste like water. Then the windshield guys showed up and replaced the windshield in a surprisingly short amount of time. Then it was time to visit Carl Strong, with whom I'd made an appointment the day before.

Carl worked in a workshop behind his home, and it was all laid out so he could produce a new frame every week. I'm still riding the replacement frame he made for me after the first one broke. The geometry is the best I've ever ridden, and I never get on the bike without thinking about what a great ride it is. It was nice to see the birthplace of my ride, and Carl and I chatted about cycling and the state of the industry a little bit, and I tried to convince him that touring without support in the European Alps is the thing to do.

I picked Carl's brain about where to go next. Like Douglas Wiegley, he suggested the Sawtooth Mountains since Glacier National Park was way too far to go. He provided several strong suggestions, including to gas up before entering the Wise river valley, which proved to be valuable advice. He also suggested visiting the Montana State University's Museum of the Rockies before leaving Bozeman, which we did.

It's a good thing we did, since my wife called while I was at the museum asking me to perform a series of transactions which involved fetching documents, finding a notary, signing it, and then sending those documents back via Fedex, which would have been much harder to do anywhere else. I felt like an RPG character on a fetch quest walking around in the 90 degree heat, but the flip side of that was that everywhere I went, everyone was very sympathetic and bent over backwards to help me. Even the notary public waived her customary fee when she saw my condition.

After that, it was time to go, and we did exactly as Carl Strong suggested, gassing up the RV at the precise gas station he told us to before heading into the Wise river valley. Along the way, we got to see the Big Hole National Battlefield, which depicted the war between the Nez Perce and the USA.

Then it was a long drive, but the place was beautiful and the road anything but boring. It was even more fun sitting in an AC cab than outside in the heat, so we only stopped at 5:00pm at the North Fork village campground which had nice river access and full hookups.

Tetons/Yellowstone RV Trip Day 10: Bridge Bay to Bozeman Hot Springs KOA

I woke up determined to beat the traffic jam and get to Mammoth Hot Springs before the crowds, so we managed to leave before 7:30am.

Arriving at Mammoth hot springs at 9:00am, we managed to see most of the sights there under cloudy skies and light rain before most of the tourists arrived, and then headed back down to boiling river for a perfect busk in the hot springs before leaving the park.

We headed out the park and upon the advise of one of the folks we met at boiling river, headed straight for Livingston and Mark's In & Out Burger. It was cheap and excellent, and then we drove towards Bozeman.

Because of the huge amount of construction in Yellowstone, our windshield had cracked. When I called the rental company, they were singularly unsympathetic, and told us that we were stuck repairing it. Fortunately, the folks we contacted in Bozeman gave us a quote and promised us a repair the very next morning.

Since it was my mom's birthday, we went out to a steak house in celebration for dinner. It was excellently priced, with very tasty steaks. I was impressed. We ended up staying at the Bozeman KOA next to Bozeman Hot Springs, and so that night paid the discounted entrance and enjoyed another busk in the hot springs, though in a much more artificial environment than in the National Park. Having had enough of warm springs, Bowen decided that he'd rather swim around in the cool pool instead, which was fine by me.

It thunderstormed that night, but we enjoyed having power, water, and clean rest rooms for a change after "dry camping" for 3 days in Yellowstone.

Teton/Yellowstone RV Trip Day 9: Lake Yellowstone, Tower Falls, and Boiling River

The weather looked nice, and the forecast was for poorer weather the next day, so we checked out a motorboat and went on Lake Yellowstone.

The boat they gave us was superfast, and we made it all the way across the Lake to Steamboat Springs at warp speed, and returned the boat in time to get in the RV, and head out to the Roosevelt area with Tower Falls in mind.

Starting at 10am instead of at 8am, however, meant that we faced traffic jams. So what ought to be an hour drive turned into an hour and a half, at very low speed. Tower falls was disappointing, but the area next to it wasn't, with lava flows that had solidified into what looked like straight columns chiseled out of sheer rock.

It was very hot at this point, but also slightly past noon. We had lunch with the generator running and air conditioning on the whole time but it made next to no difference. After lunch we walked around and then headed over to the Mammoth Hot Springs area, not to see the Hot Springs, but to visit Boiling River to sit in the warm baths.

It was very warm, but the Hot Springs was a lot of fun anyway, and Bowen loved it. By the time we were done it was way too late to see Mammoth Hot Springs and still avoid driving in the dark. The perpetual traffic jam ensured that it was dusk by the time we got back to Bridge Bay, but we still had sufficient like to dump the sewage and take on fresh water.

I remembered that there was a picnic area on gull point that we passed earlier that morning on the boat. We drove over there but it was already taken. Fortunately, there was a turnout just down the road that was both flat and empty, so while we didn't have picnic benches, we could eat and shower on the RV and still enjoy the view of Lake Yellowstone before returning to sleep in the canted campground.

Teton/Yellowstone RV Trip Day 8: Yellowstone's Grand Canyon

We woke up fairly early, ate a quick breakfast, and headed north towards the Canyon area of Yellowstone. I'd thought about booking tours in the area so I didn't have to lug the multi-ton RV around all the time, but the reality was that everything I was interested in was already booked up. With the giant RV, I thought I'd use most of the day to see the Grand Canyon area, and then come back to do laundry near the lodge.

We saw Bison on the way to the Canyon area. I would learn to hate the Bison, because they would create traffic jams whenever we had to traverse this section, but this early in the morning it was no problem. The Canyon area of Yellowstone looks impressive, but you really only need to see it once in your life. Seeing it a second time felt kinda boring. Yellowstone's not one of my favorite parks. (My personal favorite is Glacier National Park)

I was determined not to have dinner or shower in the campground at that awful canted site that evening, so after laundry, I tanked up the RV and we plonked ourselves right in front of the Lake Yellowstone Hotel right on the vista point near the beach. We probably ruined the view for those folks paying $600/night to stay at the hotel, but we had a flat parking lot and enjoyed views of Lake Yellowstone for the evening.

After dinner and showers, we headed back into the campground and slept canted. It wasn't comfortable, but at least our bellies were full and we were showered.

Teton/Yellowstone RV Trip Day 7: Colter Bay to Bridge Bay

After my memories of prior visits to the Teton National Park, the actual sunrise was comparatively disappointing. A fog bank had settled in before the Teton range, obscuring our views of the mountain and denying me the opportunity to shoot the alpenglow. I tried as best as I could but having only the Canon S100 with me and not having ideal conditions made it difficult to extract a photo I would be happy with.

After breakfast, we moved the RV out of the park. The idea was that we could take our time on the boat while reserving a picnic area for lunch and possible swimming in the Bay. The boat rental was fairly straightforward and surprisingly generous. We had a fairly restricted area we could move the boat around in, but other than that we had complete freedom to explore provided we didn't beach the boat or damage the motor.

From the boat you get a good idea of how huge Colter Bay and the lake is, but it was fairly non-descript otherwise, though we did manage to get nice views of the mountains now that most of the fog had been burned off.

After that, we went into the picnic area, where we changed into swimming trunks in our RV and then went onto the lake to swim. It was too cold to swim for long, but Bowen in his wetsuit actually came into the water and swam around in it a few times, putting everyone else to shame.

After a picnic lunch at the gorgeous picnic area, we filled up our gas and drove back into Yellowstone Park. Late in the afternoon, there really weren't many places to stop for pictures when you have a 32 foot RV. We got into Bridge Bay and had to do "dry camping" for the first time on the trip.

Pulling up the RV into the dump site in order to dump the sewer and pick up water, I was fortunate in that the folks behind us knew what they were doing and told me what I was doing wrong. There are two separate feeds into the RV for water. One of them filled the water tanks, while the other one was a direct connection into the internal plumbing for when you're hooked up. If you turn on the water pump inside the RV, that draws water from the tanks when you turn on the tap. If you're actually hooked up, what you need to do is to actually turn off the pump and let the water pressure from the direct connection feed drive the water flow. So I'd been driving around with the water tanks empty, which is no big deal since I was hooked up every night, and probably helping with my gas mileage.

In any case, we managed to get the water tanks filled up and then went to our camp site and discovered to our horrors that the site was canted: the RV listed heavily to the right, and we'd all be sleeping tilted for the next 3 days. It also made cooking and cleaning a pain in the neck. It being late in the day, I decided to bear with it for the evening and then figure out a better solution the next day.

That evening, on a walk, we saw the elk who'd been lurking around the campground.

Teton/Yellowstone RV Trip Day 6: Flagg Ranch to Colter Bay

The morning woke us up to rain and fog. It would be silly to take a boat out in this weather, but Old Faithful, Paint Pots, and the Geysers in general are immune to poor weather, so we went with the flow rather than trying to fight mother nature.

We arrived at the Old Faithful Area at 9:30, which meant that the parking lot was mostly empty and we did not experience any traffic jams. Not only could we leisurely lounge around waiting for Old Faithful to erupt, we could also look at the maps to see that Daisy was also due to erupt at a reasonable close time, so we could get 2 geysers for the price of one. I'm glad we made the effort to see Daisy, as compared to Old Faithful it's quite a bit more impressive, with an eruption time that felt significantly longer. Without the huge crowds in place, it was also a much more intimate experience. After that, we took a leisurely walk back via the various pools and geysers, and then visited the paint pots before heading towards Colter Bay.

By then, the skies had cleared some and we stopped by Lewis lake for some pictures. But what really took our breath away as we approached Colter Bay was the open views of the Tetons, now in its full summer glory.

By the time we got to Colter Bay a frenzy had over-taken me, and rather than try to find Kevin and his family to meet up, I went full-on into photo-mode and tried to find a location to shoot the sunset (and the sunrise the next morning). Colter Bay is very close to the beach, and the picnic area is easily the most picturesque in the park:

We had dinner and made plans to see the sunrise the next morning. Since we'd already seen Old Faithful et al, we weren't in a particular hurry to reach our campground the next day, so I proposed that we spend most of the day at Colter Bay rather than trying to reach the Bridge Bay campground the next day.

Teton/Yellowstone RV Trip Day 5: Jenny Lake

We got up early, had a quick breakfast, and walked over to the lodge to pick up the shuttle, which turned out to be a tiny van. I asked the driver if he had issues with having to reject people because the van was so small, and he said that the company sized shuttles to expected demand: so the van was small in the morning but the evening bus would be huge.

I'd originally planned to rent a rowboat or motorboat to fool around on Jenny Lake before hiking over to hidden falls. But it turned out that they wouldn't let a 3 year old onto one of those rented rowboats, so I nixed my original plans and got on the ferry to Hidden Falls trailhead instead, where we hiked up to the falls (which I'd never seen before), which were quite impressive.

From there, we had to decide whether to hike south back to the visitor center, or hike north around the lake, which would be a 5 mile hike. Since we were very early and the shuttle wouldn't show up till late, we decided to hike North to get a much less crowded hike. The side trips (Inspiration Point or Cascade Canyons) were ruled out given the general condition of the rest of my party.

The hike was pretty, and flat most of the way, granting us views of the Tetons, but at the headwaters of the lake, there was a steep uphill which pretty much knocked out most of the party. I also had to carry Bowen, which I didn't mind doing since he'd pretty much hiked the entire Hidden Falls trail, which was a 2 mile round trip, and did quite a bit more.

I looked at a map and realized that the northern end of the lake was very close to the Jenny Lake Lodge, which cut about 2 miles off the hike, so we hiked there and waited at the lodge instead for the shuttle. Since we were way too early for the shuttle that would take us back to Flagg Ranch, we elected to take the shuttle to Colter Bay instead, where I investigated the possibility of a motorboat rental the next morning. It seemed feasible, and we'd be staying at Colter Bay the next day anyway, so that seemed like the thing to do.

Teton/Yellowstone RV Trip Day 4: Fireside Resort to Flagg Ranch Headwaters Campground

The day looked clear, so I ditched all my existing plans and decided to take the family up to Jackson's mountain top ski resort, since the gondola ride would probably result in views. I tried to persuade Kevin to join us, but he had his heart set on going to Yellowstone's Old Faithful. I told him that the attraction was over-rated and that driving from Jackson there would likely result in a traffic jam, but he'd locked on target as they say.

The gondola ride was only 10 minutes long, but the views at the top were pretty good, though marred by a persistent low cloud that usually hid the valley below us from view. Nevertheless, we walked around marveling at the sights. We were prepared for the cold, but not for the wind, which was quite strong.

We descended the Gondola, and were told that our tickets were good for the chair-lifts as well. The chair lifts granted us a slightly lower but no less fun view, and then we had to go. I'd visited the ski resort thinking that I could just drive into the national park and visit Jenny Lake on the way to Flagg Ranch. However, that entry into the park was closed to RVs, and so we had to go all the way back to Jackson and then drive North from there. This delayed us and made a visit to Jenny Lake unfeasible for the day, but I decided to instead visit every sight along that entry.

The entry was filled with all the famous overlooks of the Tetons, from the Snake River overlook to Schumacher Landing Road, as well as the Oxbow bend. We took our time and stopped at every viewpoint to take photos, ending up in Colter Bay just before 4:00pm for the kids ranger program.

From there, we drove up to Flagg Ranch, where the campground office helpfully told me that the park actually had a shuttle service between Flagg Ranch and Jackson, with multiple stops in between at various attractions. Not only was the pick up convenient, but it was also free for campers at Flagg Ranch!

I took a look at the schedule and realized that we could use it to visit Jenny Lake the next morning, at the expense of having to get up pretty early to take advantage of the shuttle. This was not an issue for us, since we were good at getting up early at this point, so we resolved to leave the motorhome parked at Flagg Ranch the next day.

Kevin showed up with his RV, and told me that as predicted, he faced heavy traffic and found Old Faithful over-rated. I told him about the free shuttle service and tried to convince him to stay at Flagg Ranch and join us, but he had his heart set on Colter Bay the next day, and was determined to show up there early to get a first-come-first-serve camping spot.

Review: Foundation's Triumph

Foundation's Triumph is written by David Brin, which was the main reason I picked up the book despite being underwhelmed by Foundation and Chaos.

The book details Hari Seldon's last adventure, where on a whim, he leaves Trantor on a hunt to figure out why the soils of the various planets of the galaxy are clustered the way they are. As a result of this adventure, Seldon discovers the robotic forebears of the human pioneers who settled the galaxy, and sets off a struggle between various factions of robots, representatives of the Chaos world Klinta, and of course, Seldon's own conflicting feelings about his Foundation and Second Foundation.

The net result is a bit of a mash. You do get a nice complex plot with lots of moving parts, but a shortage of new ideas which is what David Brin's famous for. There's a sense that Brin's far too constrained by having to work within Asimov's universe, as well as the issues of writing a prequel: there's too much already known about the future, and not enough freedom to introduce new concepts.

Ultimately, a disappointing read.

Teton/Yellowstone RV Trip: Day 3: Rafting on the Snake River

The rafting trip was broken into 2 parts: the scenic tour in the early morning, where everyone was allowed to go, and the late morning white water trip, where Bowen was excluded and my father elected to stay behind to watch him.

The morning trip was indeed a slow float down the river, where everyone stayed dry. Our river guide, a 27-year old woman from the East Coast, was formerly a pre-med before deciding that it wasn't for her and came to Jackson to lead an outdoor life as well as retrain for education.

We spotted 10 bald eagles, multiple ducks, and Bowen got a chance to row with the guide. I expected a struggle to keep him off the white water trip, but it turned out that an early explanation had stuck, so he agreed to stick around the rafting office and take care of Grandpa.

The white water trip was at most a class 3, since it was summer and the water was low. It was fairly tame by white water standards, but it was a lot more active than the earlier trip, and we got wet quite often.

There was even a section where you could jump off the boat and swim, and the weather being warm enough to do so this time, quite a few folk took the river guide's offer.

After that, we visited the Pearl Street Market to buy lunch, and then were driven back to the Fireside resort. We spent the rest of the evening doing laundry.

Teton/Yellowstone RV Trip Day 2: Bear Lake State Park to Fireside Resort

We got up early the next day to make the remaining drive to Jackson, Wyoming, where I'd pre-booked a site at the Fireside resort for 2 days. I'd gotten to realize early on that a 32' RV was hard to get a space for, and so had booked about the first 9 days of the trip.After that, I figured I'd be competent to fly by the seat of my pants.

The reason for staying here two nights was because I'd signed us up for a rafting trip the next day. We'd arrived early enough to be able to take the bus into town and walk around, so we did so. We then had an early night so we could wake up in time for the pickup from the rafting company the next day.

Tetons/Yellowstone RV Trip Day 1: Salt Lake City to Bear Lake State Park

In early April, we thought it might be a good idea to take the family to the Tetons and Yellowstone for some RV camping. Because of the number of people involved, I thought that an RV might be practical. I've never tried it before, but given my experience with relatively large sailing catamarans, I didn't think that maneuvering an RV would be especially difficult.

Because of the number of people involved, I was forced to reserve a fairly large motorhome. I ended up picking a class B Sunseeker 3170 from Utah RV Rentals.

Because of a few last minute change of plans (driven entirely by external circumstances beyond our control), neither Boen nor Xiaoqin could come, so my parents were recruited to come along instead.

On August 3rd, we flew a 6:15am direct flight from San Jose to Salt Lake City, where upon landing the representatives from Utah RV Rentals met us, tossed everything into an open-top pick up truck, squeezed all 6 of us into the cab, and drove us to pick up the RV. You might think me insane for deliberately picking an absurdly early hour, but my reason for doing so came from my experience with sailboat charters: I expected the check-out to take 2 hours, and then wanted another couple of hours for provisioning, and then to gingerly make my way to Bear Lake State Park, where I had a fully hooked up connection.

The check-out doesn't take nearly as long for an RV: you're presumed to know how to drive a car. The systems aren't nearly as complicated, and RV rental companies seem to be really casual about teaching you how to operate systems. For instance, on a sailboat, the propane is usually cut off whenever you're sailing, but on an RV it's apparently OK to turn on the propane while driving!

We left the RV rental place around 11:00, made our way to a Costco, and then proceeded to eat lunch before buying about 10 days worth of consumables. The RV handled pretty nicely, much closer to a boat than a car, but much less bulky and more maneuverable than any sailboat I've had to drive. What caused us to drive slowly, however, was an unusual storm, which made visibility poor and traffic on I-15 slow an uncharacteristically slow 45mph. We did leave before rush hour traffic, so we were able to get onto the smaller roads past Ogden fairly early.

If you use Google Maps to visit Bear Lake State Park, it'll take you to the wrong one. What you want is the Rendezvous Beach campground. We got there at around 5:00pm, proceeded to set up camp and dinner, when Kevin arrived. The kids got to play, and we got to walk around in the park, which wasn't particularly pretty, but it was only a temporary stop.

As expected, Bowen loved RV camping, and after playing was happy to sleep in the same space as me above the cab of the RV. He kicked around all night, however.