I originally published this in May of last year in Idea Collector, one of my old blogs.
Epistemic status: This is a quick write up of my personal experience wandering Osaka, Kyoto, and Tokyo last May. I'd be curious to learn how it compares to objective measures, models, others' experience, and so on!
Epistemic effort: I noted my impressions in bullet-point form while wandering around the cities, then when I returned from the trip I spent ~1h turning those messy notes into a form that might actually make sense to a non-Devon.

The continuity of urbanism along the 500km shinkansen bullet train track between Osaka and Tokyo was astonishing. Throughout the journey, the buildings almost never thinned out, and at no point did I feel like we were in suburbia, let alone true countryside. The route went through Kyoto and Nagoya, both major cities in their own right, and there was a continuous metropolitan zone along the entire shinkansen line.

At the same time, I was surprised by Japan’s lack of density, especially in Tokyo. I didn’t find any one area that felt especially built up. The entire metropolitan area is highly populated, and the median building height is greater than most American cities, but there was no central urban core with significantly higher density than the surrounding areas. Instead, the height level of buildings was mostly continous across all neighborhoods. By contrast, major American cities tend to have an extremely dense, bustling center with steadily decreasing density in neighborhoods radiating out from that core. This is usually a business and shopping district with high rise apartments. Many people live in that center, but even more commute in and out each day for work.

Japanese cities’ lack of centers with extreme density surprised me. My intution about how cities formed was a life-scale Hotelling Game, in which the optimal location for a new enterprise is in the middle of where everything else is. This minimizes the distance to the greatest number of amentities, shops, people, and so on, and so it is the highest value land. Businesses want to be where people are, and people want to live close to their schools and work. This creates a cycle in which city centers are valuable, incentivizing developers to build denser, taller buildings. As more people move in, the land in those centers because even more valuable, which incentivizes more building. With all these uses vying for the same space, the only alternative to build up, and you get urban cores with a far greater average building height than the surrounding areas.

The above diagram is from Alain Bertaud's paper The spatial organization of cities. My simplistic mental model of cities as a Hotelling Game matches "A. The classic mono centric model", but Tokyo feels closer to C. or D.
Of course this is a simplification—zoning rules, geography, weather, and more affect land values, too. But the Hotelling dynamic plays a role in shaping the density and land values of cites. You can see the effect in the Google Earth images of New York and San Francisco below.
But Tokyo doesn't follow this model. The entire metro region is one of the most populous in the world, but it's not actually that tall. There are lots of "centers" of the city, each of which have tall buildings, but almost none are extraordinarily tall. This is directly contrary to the impression I'd had before visiting the city. I feel like most imagery of Tokyo is dense, bustling, neon-lit streets, but it was much more human-scale and personable than that. Moreso than the much less dense San Francisco, in my opinion.

The Financial District on the south end of the island and Midtown just below Central Park are extremely dense. Concentration of people draws in businesses and shops, which in turn makes it a more attractive place to build housing, transit, and amenities. Meanwhile, building height drops off quickly outside of these centers. The median number of floors in Greenwich Village (around Washington Square Park) is less than four!

San Francisco is a striking illustration of this phenomenon. It’s is a bit unfair, because most in neighborhoods outside of the Financial District and (as of 2015) Mission Bay, buildings are limited to 4-6 stories. However, the SF’s neighborhoods for the most part took on their current form before building codes cemented it, so Hotelling still provides a useful model.

Japanese cities don’t appear to follow this logic. Tokyo’s tall buildings are fairly even distributed throughout the city. There is variation of course, and neighborhoods like Ueno are more spread out than others, but there is no single area that is far taller than the rest. The closest thing to "downtown" was the area around Tokyo Station, but that was more a function of its role as an intercity transit hub connecting the shinkansen lines than of being the central business district.

I’m still at a loss as to what caused this dynamic. My best guess is that the city developed alongside a mature, well-supported rail system that could whisk people from one part of the city to another, so the advantage of being in the middle of it all was lower. Manhattan’s subway system could be described in much the same way though, so this answer is not satisfying.

Japanese land use regulations are also very different from their counterparts in the US , so that certainly played a part too. It’s possible that this accounts for most of the difference. In the San Francisco most neighborhoods limit buildings to 4-6 stories, which artificially enforces the funnel shape, where the old parts of the city that were dense before regulations took hold are the only place where tall buildings are allowed in the first place. Other US cities follow a similar regulatory pattern. This raises an interesting question — if SF lifted all density limits, where would we see growth? Would the financial district grow even taller, or would the surrounding neighborhoods build out to match its height? My guess is it would be a mixture of the two, with further densification of the core as well as development of the surrounding areas, with lessening growth as you went outward. Overall though, the effect would likely be an evening of heights across the city. I’d guess that the variance in median heights between neighborhoods would decrease, and we’d have a situation closer to that of Tokyo.

I’m not well-versed in the history of Japanese land use. So if you have any other ideas about why their building height variance is low, please let me know!