Wednesday, December 30, 2015

It's not just a game, ...

... it's a game that makes you feel like you're not playing a game.

‘The Witcher 3’ Understands War

Many video games are power fantasies, and most that involve warfare depict the glory of combat and put the player in the lead role. Not so in The Witcher 3. Geralt has his own motivations, and he does his best to avoid politics and the larger conflict between Nilfgaard and the Northern Kingdoms.

And CD Projekt never depicts war as glorious or fun. Soldiers describe combat as a lot of boredom and waiting punctuated by moments of frenzied madness. The Northern War of the The Witcher 3 is all about waiting, survival and boredom.

Oracle v Google speculation abounds

The chatter over at Hacker News picks up this: Mysterious Android codebase commit

The discussion goes on for pages and pages, in typical Hacker News style.

The most interesting observation, I think, is this one: This diff is more explicit about what's going on

Change dependency from libart -> libopenjdkjvm.

LibART, I think, is the "Apache RunTime", also known as "Harmony", about which you can read more here: Apache Harmony, while you can read about OpenJDK at the corresponding OpenJDK page.

An article over at VentureBeat offers its own speculation: Google confirms next Android version won’t implement Oracle’s proprietary Java APIs

“As an open-source platform, Android is built upon the collaboration of the open-source community,” a Google spokesperson told VentureBeat. “In our upcoming release of Android, we plan to move Android’s Java language libraries to an OpenJDK-based approach, creating a common code base for developers to build apps and services. Google has long worked with and contributed to the OpenJDK community, and we look forward to making even more contributions to the OpenJDK project in the future.”

I haven't been paying a lot of attention to the case this fall, and I haven't seen a lot of coverage, either, so these random speculations intrigued me, though I have no idea what they mean.

Tuesday, December 29, 2015

Here's some good news with which to end your 2015 ...

... Guinea Declared Free of Ebola Virus That Killed Over 2,500

Guinea was declared free of Ebola transmission on Tuesday after more than 2,500 people died from the virus in the West African nation, leaving Liberia as the only country still counting down the days until the end of the epidemic.

The announcement comes 42 days after the last person confirmed with Ebola tested negative for a second time. The country now enters a 90-day period of heightened surveillance, the U.N. World Health Organization said.

UPDATE: A nice follow-up by the U.S. Government, surveying all the work done, and all the work left to do.

Saturday, December 26, 2015

Pillars of Eternity is my new addiction

I've been keeping my eye on Pillars of Eternity for several months now, but hadn't yet taken the plunge.

Then, over the holiday break, it went on deeply-discounted sale on Steam.

So I made the decision.

And, wow, is this a great game!

It's everything the reviews said it was.

And the real-time nature, so far, hasn't been much of a problem. The first thing I did was to find the Options, and in there find the Auto-Pause options, and there was a conveniently-labelled checkbox: Set All.

So I checked it, and it set all, and so far I've spent 10 hours exploring this new world.


Wednesday, December 23, 2015

Cow algorithms

I loved this little interlude from Brian Harry's blog: Is your stomach bigger than your eyes?

As it turns out calves have a simple algorithm for navigation – If my head fits through it, I can go. The problem is that their shoulders are wider than their head and their hips are even wider yet. In fact, this behavior is one we take advantage of. We catch cows in a “head gate” when we need to work with them. The cow walks down a chute, sees a gap in the head gate and tries to go through it. The edges of the gate catch their shoulders and close, locking the cow in.

The complexities of health-care costs in America

The New York Times continues its excellent work on trying to investigate and explain the complexities of health-care costs in America with this fascinating piece: The Experts Were Wrong About the Best Places for Better and Cheaper Health Care.

Health care researchers who have seen the new findings say they are likely to force a rethinking of some conventional wisdom about health care. In particular, they cast doubt on the wisdom of encouraging mergers among hospitals, as parts of the 2010 health care law did.

Larger, integrated hospital systems – like those in Grand Junction – can often spend less money in Medicare, by avoiding duplicative treatments. But those systems also tend to set higher prices in private markets, because they face relatively little local competition.

The article goes on to note that:

Below, a scatterplot showing medical spending per person for Medicare and private insurance for all 306 hospital referral regions in the United States.

The chart looks random, and that’s the point: There is no real relationship between spending in one system and the other.

The answers aren't easy, but the article gives lots of suggestions for further investigation, and for further thought.

And big thanks to The New York Times for continuing to chip away at this complicated yet crucial puzzle.

Tuesday, December 22, 2015

Hot Links for the Juniper story

There's lots going on with the Juniper story. Here are some pointers to get you started:

  • Some Analysis of the Backdoored Backdoor
    Alas, while Juniper used Dual_EC_DRBG with the P-256 NIST curve and the point P specified in SP 800-90A in ScreenOS — the operating system running on NetScreen VPN gateways — they chose to use a different point Q and not the one supplied in the standard for P-256.


    However, apparently starting in August 2012 (release date according to release notes for 6.3.0r12), Juniper started shipping ScreenOS firmware images with a different point Q. Adam Caucill first noted this difference after HD Moore posted a diff of strings found in the SSG 500 6.2.0r14 and the 6.2.0r15 firmware. As we can deduce from their recent security advisory and the fact that they reverted back to the old value Q in the patched images, this was a change not authored by them.

  • On the Juniper backdoor
    The creepiest thing about CVE-2015-7756 is that there doesn't seem to be any unauthorized code. Indeed, what's changed in the modified versions is simply the value of the Q point. According to Ralf this point changed in 2012, presumably to a value that the hacker(s) generated themselves. This would likely have allowed them to passively decrypt and ScreenOS VPN sessions they were able to eavesdrop.
  • DUAL_EC Question of the Day
    People assumed that the NSA wanted a backdoored random number generator so they could look at other people's traffic, but of course a plausible answer is that a backdoored random number generator is even more useful for looking at your own traffic in an economical way.
  • CVE-2015-7755: Juniper ScreenOS Authentication Backdoor
    The argument to the strcmp call is (...), which is the backdoor password, and was presumably chosen so that it would be mistaken for one of the many other debug format strings in the code. This password allows an attacker to bypass authentication through SSH and Telnet. If you want to test this issue by hand, telnet or ssh to a Netscreen device, specify any username, and the backdoor password. If the device is vulnerable, you should receive an interactive shell with the highest privileges.
  • First Exploit Attempts For Juniper Backdoor Against Honeypot
    We are detecting numerous login attempts against our ssh honeypots using the ScreenOS backdoor password. Our honeypot doesn't emulate ScreenOS beyond the login banner, so we do not know what the attackers are up to, but some of the attacks appear to be "manual" in that we do see the attacker trying different commands.

Monday, December 21, 2015

Quanta on Graph Isomorphism

I think the best casual-reader coverage of Babai's new finding, so far, is this article on Quanta: Landmark Algorithm Breaks 30-Year Impasse.

One of my favorite parts is the section where the author discusses Babai's patience and perseverance:

Babai’s proposed algorithm doesn’t bring graph isomorphism all the way into P, but it comes close. It is quasi-polynomial, he asserts, which means that for a graph with n nodes, the algorithm’s running time is comparable to n raised not to a constant power (as in a polynomial) but to a power that grows very slowly.

The previous best algorithm — which Babai was also involved in creating in 1983 with Eugene Luks, now a professor emeritus at the University of Oregon — ran in “subexponential” time, a running time whose distance from quasi-polynomial time is nearly as big as the gulf between exponential time and polynomial time. Babai, who started working on graph isomorphism in 1977, “has been chipping away at this problem for about 40 years,” Aaronson said.

Babai's paper is up on Arxiv: Graph Isomorphism in Quasipolynomial Time. Although this is definitely not for the casual student, it's a remarkably clear paper, systematically working its way through the problem in detail.

As Babai notes near the end of the paper, in this area of Computer Science, theory and practice have taken different paths:

The purpose of the present paper is to give a guaranteed upper bound (worst-case analysis); it does not contribute to practical solutions. It seems, for all practical purposes, the Graph Isomorphism problem is solved; a suite of remarkably efficient programs is available (nauty, saucy, Bliss, conauto, Traces). The article by McKay and Piperno [McP] gives a detailed comparison of methods and performance. Piperno’s article [Pi] gives a detailed description of Traces, possibly the most successful program for large, difficult graphs.

The article on Traces is here: Search Space Contraction in Canonical Labeling of Graphs. The description of the backtrack construction on page 7 is accompanied by beautiful full-color diagrams; a picture is truly worth thousands of words here.

Lastly, the first of the four explanatory talks given by Babai at the U of C is available on YouTube.

Wednesday, December 16, 2015

You can't even believe in The Gray Lady anymore.

It sounds like The New York Times very badly blew it in one of their lead stories in last weekend's edition: The FBI just blasted reporting on the San Bernardino killings.

If this is indeed accurate, shame on The New York Times.

And shame on the other publications who simply parroted that article as though it were fact.

And kudos to those who have been exercising patience, waiting for investigators to sift things through and try to give a more reasoned analysis of matters.

And general Yuck! to the Internet for pushing the idea that the latest information is the greatest information.

Slow down, world, and take time to think.

When 768 conflicts equals zero conflicts

While reading the completely fascinating Move Fast and Fix Things, I came across this spectacularly delicious tidbit:

A much more interesting case happened when a merge that was clearly a conflict in libgit2 was being merged successfuly by Git. After some debugging, we found that the merge that Git was generating was broken — the single file that was being merged was definitely not a valid merge, and it even included conflict markers in the output!

It took a bit more digging to find out the reason why Git was "successfully" merging this file. We noticed that the file in question happened to have exactly 768 conflicts between the old and the new version. This is a very peculiar number. The man page for git-merge-one-file confirmed our suspicions:

The exit value of this program is negative on error, and the number of conflicts otherwise. If the merge was clean, the exit value is 0.

Given that shells only use the lowest 8 bits of a program's exit code, it's obvious why Git could merge this file: the 768 conflicts were being reported as 0 by the shell, because 768 is a multiple of 256!

One of the things about rewriting an algorithm is that you have to understand whether a difference in behavior represents a bug in your new code (which is the vastly more likely case), or is actually a discovery of a bug that existed in the prior implementation, but was hitherto unknown.

Big kudos for the libgit2 team for working their way through this one!

Monday, December 14, 2015

More words we haven't heard in a long time...

Winter Storm Echo Spreading Heavy Snow From West to Upper Midwest

In California's Sierra Nevada, peak snow amounts of 20 inches were reported at Sugar Bowl Ski Resort and at Tahoe Donner Ski Area.

On the Nevada side of the Sierra Nevada, Mount Rose Ski Area received 20 inches of snow. Up to 24 inches of snow was recorded just northeast of Incline Village at an elevation of 6,700 feet.

It's not just a game, ...

... it's a holiday-themed contest with an awesome first prize!

This time we will pick only one winner who will receive a one-of-a-kind custom made Witcher sword! Check out the photo of the blade below. This thing was forged by Hattori himself (seriously)! Please check if we can send the prize to your country -- there might be some legal restrictions regarding sending sharp objects!

Saturday, December 12, 2015

In which people discuss things I don't understand

Tis the season for the sharing economy...

  • The Airbnb Endgame
    What all of this posturing about bad hosts is meant to obscure is that the exact kind of hosting that Airbnb does want, what it envisions as the core of the service—thousands and thousands and thousands of regular people sharing their homes whenever they’re not home—is largely illegal in New York. Like actually against the law, up-to-$5000-fine kind of illegal! While it’s not uncommon for startups looking to “disrupt” something to begin operating in liminal legal spaces, the laws in New York surrounding short-term rentals are rather unambiguous, despite Airbnb’s protests that there is a “lack of clarity” around the rules. It’s illegal for a person living in an apartment in a “multiple-dwelling”—essentially, a building with more than two apartments—to rent out his or her entire home for a period of less than thirty days if he or she is not present. (It’s perfectly legal in single- and two-family homes, which aren’t “multiple dwellings.”) So, a whole-home listing in a building with more than two apartments that doesn’t have a minimum stay of thirty days—and isn’t a hotel or boarding house or some such—is probably illegal.
  • How Uber cleverly controls its stock so it won't have to go public anytime soon — unlike Facebook, Twitter, and Google
    Even though Uber is almost the same size and scale as Facebook was back then, it's in a substantially better situation. The company has been careful to learn from Facebook's missteps and control who owns its stock so it isn't forced into an IPO.
  • Uber seeks to head off lawsuits with new binding driver agreement
    All 400,000-plus drivers cannot receive any ride requests until they accept the agreement, which lays out a lengthy provision requiring mandatory arbitration starting on page 15 and flagged on the first page. While it includes a way to opt out, many drivers may not understand that or may fear retaliation for doing so.
  • Uber doesn’t want drivers to sue again, so it pushes them to arbitration
    A recent investigation into the general arbitration system by The New York Times found that private arbitration is subject to little oversight, rarely can be appealed, does not have clear evidence rules, and is subject to rampant conflict of interest that tilts towards corporations.
  • Uber wants drivers to sign new lawsuit pledge
    Uber says the new driver agreement was necessary because on Wednesday U.S. District Judge Edward Chen ruled that part of the agreement Uber drivers had been signing was not enforceable, rendering the entire agreement unenforceable.

    In order to correct that, Uber rewrote the agreement and removed a requirement that arbitration be confidential. The company informed Chen of the new agreement on Thursday and pushed it out to drivers Friday, the San Francisco-based company said.

  • Uber Is Said to Be Shaking Up Policy and Communications Team
    Uber brought in Rachel Whetstone, a top Google policy and communications executive, to lead Uber’s overall policy and communications. Ms. Whetstone hired Jill Hazelbaker, an executive at Snapchat and a former colleague of Ms. Whetstone at Google, where Ms. Hazelbaker also ran policy and communications teams.

    It appears, insiders say, that the company is consolidating its communications and policy operation under its new leadership from Google.

My philosophy on saving for retirement

Due to an (unexpected, involuntary) change of employment about 6 years ago, I found myself with a 401K account that I was required to rollover into an IRA, and so I became a more active participant in my retirement planning.

Anyone who, like me, got started in trying to manage his own retirement account at the start of 2010 surely thinks of himself as the greatest investment analyst in the history of the world, when the actual fact is that I simply got in on the best 5 years (2010-2014) that the market ever had, or ever is likely to have.

During that time, I bought and held a small number of Amazon shares. I also got lucky enough to buy Netflix stock when it took a BIG dip, and then held some of those shares through its 7-way split. And I lucked into buying Hawaiian Airlines at a time when all the airline stocks were in the tank due to the Great Recession.

So pretty much all my gains during those 5 years were in those three stocks, and they were all lucky choices to which I committed relatively small investments.

In each case, when they started to race up I sold some of my shares, enough to cover my initial investment, and let the rest ride.

I'm a HUGE fan of what they call "dollar cost averaging", which is basically: don't invest all at once; don't sell all at once. Instead, invest gradually over a period of time, splitting your purchase into separate smaller purchases, or splitting your sale into separate smaller sales. That way, if you have the bad luck to pick a bad day for one of your orders, the probabilities are that the luck will even out on the other ones.

And, I ALWAYS ALWAYS ALWAYS use limit orders, for both buying and selling, never market orders. For buying, I pick a target price which is slightly (1-3%) lower than the current price, and let the computers monitor it and execute on a price dip. And for selling I do the same thing.

So the computers do the hard work.

Since my IRA company has a per-trade commission, I need to be careful not to execute too many orders. But in practice I only issue about 1-2 orders a month, so I'm not spending much on commissions.

Even though Amazon and Netflix are incredibly volatile and incredibly pricey, they are also extremely-well-run companies with huge potential ahead of them. I think this is also true of Microsoft, Intel, and Google.

Tesla is also a fascinating company, but I don't own it.

The majority of my portfolio is, and has been, in "consumer staples": I own Clorox, Johnson & Johnson, Procter & Gamble, Church & Dwight, Campbell's, Kimberly Clark, VF Corp, Wal-Mart, etc. Over almost any period of time (1 year, 5 years, 25 years, 70 years), these companies have been steadily growing at a slow rate.

AND, they pay dividends!

Microsoft and Intel also pay dividends. I like companies that pay dividends. Amazon and Netflix stand out as exceptions; pretty much all my other investments are in dividend-paying companies.

I have some other investments in other companies, but much of that is a motley mess.

I don't spend a lot of time on my portfolio. I tend to check it once a week or so. It's really important not to watch my portfolio, because many of my holdings are quite volatile and my account routinely goes down and up by significant amounts in a single day, so if I watched it every day I'd die from the emotional roller-coaster of it.

So I just let the various dividends accumulate in my "cash bucket" in my account, and, every so often, enough has accumulated that I re-invest those dividends by buying some more of one of the stocks I already own (or, VERY rarely, initiating a position in a new company that I like). The re-investment decision is probably really important, but I don't spend much time on it; I just pick a company that I "want" to own some more of, and enter a buy order for the cash that I've got available to invest.

So I stay mostly fully invested.

I have about 12% of my portfolio in bonds, which has been a complete disaster, since the last 7 years have been a disaster for bonds. Now, since I'm just talking about my IRA here, and I also have a 401K at my company which is 100% invested in stock funds, the actual overall percentage of my retirement savings is even more tilted to stocks than bonds, so I'm really only holding like 8% of my portfolio in bonds.

But then, I'm still (relatively) young, and am hoping to go at least 10 more years before I start to draw on this money, so staying fully in stocks for now is a good strategy. When I get to my mid-60's I'll probably start moving some of that money over to bonds, though I still love those consumer staples and their dividend rate remains better than any bond fund I've seen out there.

I really don't do much with stock screeners, research, etc., other than reading the business pages once in a while. Every so often I hear about a company that I'd like to own, and then I expand my portfolio, but really I have enough separate investments right now that I don't really want any more things to look at.

In fact, basically every company or mutual fund that I bought after spending lots of time doing screens and research was a failure for me. I did much better just picking companies I like and investing in them gradually over time.

The one and only financial website I recommend is

Pretty much everything Barry Ritholtz has on his site is superb and I read that site (and the things he links to) regularly.

I am a long-term investor: the bulk of my new investment is in my 401K at my company, which unfortunately is a very poorly run investment operation (small fund selection, high fund fees, high administration fees), but it is highly tax-advantaged so that's what I'm doing. And my company does a small regular contribution to my plan. So, overall, the bulk of my new investments are going there, and my IRA is just sitting there gathering dividends and slowly increasing its holdings of my existing companies.

I'm hoping that, in 10-12 years when I retire, I'll have enough. But I'm not really sure that I will.

But I don't think there's much I can really do about that worry, other than to keep on with the current plan.

My back-of-the-envelope math says that my continued 401K contributions, combined with what I've already saved, and the paltry amount of Social Security that I'll qualify for when I turn 70, will be adequate.

So I try to mostly not think about this stuff, because it's depressing and I have many other things to do.

But somebody asked, so I replied.

And I figured I'd put it on my blog, because maybe somebody will tell me that I've totally overlooked something.

Wednesday, December 9, 2015

It feels like it's been forever since we heard these words...

... Pacific Northwest Storm Parade to Bring More Rain, Wind and Mountain Snow Through the Weekend

A classic November-December setup featuring a powerful jet stream stretching from eastern Asia across the Pacific for 5,000 miles to the Pacific Northwest is acting as the conductor for this storm parade. The persistent pipeline of moisture is being supplied by what meteorologists sometimes refer to as an atmospheric river. In this case, the plume of moisture impacting the Northwest extends all the way from the western Pacific Ocean near the Philippines.

Will events play out as predicted?

New database systems Red Book!

Well, this is exciting: Readings in Database Systems, 5th Edition.

Readings in Database Systems (commonly known as the "Red Book") has offered readers an opinionated take on both classic and cutting-edge research in the field of data management since 1988. Here, we present the Fifth Edition of the Red Book — the first in over ten years.

I had the original edition, got it in 1989 if memory serves. Chewed it to death.

The arrival of Transaction Processing: Concepts and Techniques somewhat obsoleted the Red Book, but I still have wonderful memories of the original edition.

Now I have something new to chew!

fsync and Virtual Machines

Oh, the wonders of low-level systems software.

The documentation for fsync() says:

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) so that all changed information can be retrieved even after the system crashed or was rebooted.

But what if you are running in a Virtual Machine?

An interesting article says:

Hypervisors outperforming the host machine is the most interesting to me. The results of this test clearly show that the hypervisors must be lying about synced writes for performance. This corroborates what I’ve seen with Packer as well, where if the virtual machine is not cleanly shut down, committed writes are lost. fsync() in a virtual machine does not mean that the data was written on the host, only that is is committed within the hypervisor.

And yet, another interesting article says:

ESX(i) does not cache guest OS writes. This gives a VM the same crash consistency as a physical machine: i.e. a write that was issued by the guest OS and acknowledged as successful by the hypervisor is guaranteed to be on disk at the time of acknowledgement. In other words, there is no write cache on ESX to talk about, and so disabling it is moot. So that’s one thing out of our way.

Is this more than a "he says, she says" thing? That is, is there a more definitive resolution of this question somewhere?

Note, btw, that the core behavior of Linux's fsync() command is not even clear, putting aside Virtual Machines:

By durable, I mean that fsync() should actually commit writes to physical stable storage, not just the disk write cache when that is enabled. Databases and guest VMs needs this, or an equivalent feature, if they aren't to face occasional corruption after power failure and perhaps some crashes.

I'm sure there are places where this information is definitively and clearly documented.

I'm just not sure where those places are.

Sunday, December 6, 2015

Time passes

I signed up for Windows 10 on my computer back in, what, June?

Around the end of July, the Inter-webs were full of people discussing their experiences.

Windows 10 wasn't ready for my computer.

In late September, I fussed with it some.

I dug into the Windows Update history screens, and there were strange event codes and unfamiliar messages.

Researching them with various search engines lead to lots of people with similar strange messages, and odd suggestions to do things like "clear your download directory, maybe you had a corrupted download."

I tried a few of those strange suggestions, but basically forgot about things, and determined that the most probable outcome was that this computer was only going to run Windows 8.1 (which is fine), and maybe in the future I might try running Windows 10.

Then, Friday, the computer seemed to wake up, and announced that it thought that Windows 10 was "coming".

And, today, it asked me if I wanted to start the download.

So, we'll see.

Maybe Windows 10 is in my future.

Maybe not.

And (is it an omen), just as I go to try to post this, Blogger is down.

For the first time in I-don't-know-how-long.

Time passes.

Friday, December 4, 2015

It's not just a game, ...

... it's GOTY 2015: The Witcher Takes it, But Should it Have Been Fallout 4?

There is something about The Witcher that is so eerie, spooky and dramatic. I can’t quite put my finger on it. Whether it’s playing as a Van Helsing-esque mutant, or the monsters themselves that you are hired to hunt. Whatever the driving force might be, there is something truly special about CD Projekt Red’s Witcher 3. Again, as far as games go, I had a fair many gripes with the title, but racking up a total play time of over 400 hours, it’s safe to say I got my money’s worth. The other thing is that The Witcher wasn’t, like Fallout, merely substance. The Witcher was art. The beauty, the immersion, it was all so well done that it felt less like a game than a film. It was truly an amazing experience. Fallout 4 is a fantastic game, and an unusually charming one, for the amount of bugs that players face each time they boot it up.

Stuff I'm reading, early December edition

Wow! Go away on vacation for a week and then have a busy week at work and my, what a lot of stuff accumulates in the in-box...

  • The Moral Character of Cryptographic Work
    Cryptography rearranges power: it configures who can do what, from what. This makes cryptography an inherently political tool, and it confers on the field an intrinsically moral dimension.
  • A Little More on the Graph Isomorphism Algorithm
    The key ideas of the algorithm for GI are really classic ones from design of algorithms. The genius is getting them all to work together. The ideas break into two types: those that are general methods from computer science and those that are special to the GI problem.
  • The best books of 2015
  • RTS AI: Problems and Techniques
    Real-Time Strategy (RTS) games is a sub-genre of strategy games where players need to build an economy (gathering resources and building a base) and military power (training units and researching technologies) in order to defeat their opponents (destroying their army and base). Artificial Intelligence problems related to RTS games deal with the behavior of an artificial player. This consists among others to learn how to play, to have an understanding about the game and its environment, to predict and infer game situations from a context and sparse information.
  • DRAM’s Damning Defects—and How They Cripple Computers
    In computing systems built on such huge scales, even low-probability failures take place relatively frequently. If an individual computer can be expected to crash, say, three times a year, in a data center with 10,000 computers, there will be nearly 100 crashes a day.

    Our group at the University of Toronto has been investigating ways to prevent that. We started with the simple premise that before we could hope to make these computers work more reliably, we needed to fully understand how real systems fail. While it didn’t surprise us that DRAM errors are a big part of the problem, exactly how those memory chips were malfunctioning proved a great surprise.

  • Challenges of Memory Management on Modern NUMA System
    Modern NUMA systems are quite different from the old ones, so we must revisit our assumptions about them and rethink how to build NUMA-aware operating systems. This article evaluates performance characteristics of a representative modern NUMA system, describes NUMA-specific features in Linux, and presents a memory-management algorithm that delivers substantially reduced memory-access times and better performance.
  • SGX Hardware: A first look
    Without much fanfare, Intel has released Software Guard Extensions (SGX) in Skylake.
  • Experimental Security Analysis of a Modern Automobile
    Of course, it’s possible that the software on those cars could have been updated back at dealerships in the intervening years – but that wouldn’t address all of the issues in the paper, and evidence suggests plenty of vulnerabilities still exist.
  • Experimental Security Analysis of a Modern Automobile
    Over a range of experiments, both in the lab and in road tests, we demonstrate the ability to adversarially control a wide range of automotive functions and completely ignore driver input — including disabling the brakes, selectively braking individual wheels on demand, stopping the engine, and so on.
  • Fast and Vulnerable: A Story of Telematic Failures
    In this paper we examine a popular aftermarket telematics control unit (TCU) which connects to a vehicle via the standard OBD-II port. We show that these devices can be discovered, targeted, and compromised by a remote attacker and we demonstrate that such a compromise allows arbitrary remote control of the vehicle.
  • How Change Happens: Consultation draft
    How Change Happens draws on many first-hand examples from the global experience of Oxfam, one of the world’s largest social justice NGOs, as well as Duncan Green’s 35 years of studying and working on international development issues. It tests ideas and sets out the latest thinking on what works to achieve progressive change.
  • Linux Performance Analysis in 60,000 Milliseconds
    You login to a Linux server with a performance issue: what do you check in the first minute?
  • Fixing the #1 Problem in Computer Security: A Data-Driven Defense
    The implementation weaknesses described in this white paper are common to most organizations, and point to limitations in traditional modeling of and response to threats to computer security. Most of the problems occur due to ranking risk inappropriately, poor communications, and uncoordinated, slow, ineffectual responses.
  • Why Johnny Still, Still Can’t Encrypt: Evaluating the Usability of a Modern PGP Client
    Our results shown that more than a decade and a half after Why Johnny Can’t Encrypt, modern PGP tools are still unusable for the masses. We finish with a discussion of pain points encountered using Mailvelope, and discuss what might be done to address them in future PGP systems.
  • Prudent Engineering Practice for Cryptographic Protocols
    The principles are neither necessary nor sufficient for correctness. They are however helpful, in that adherence to them would have avoided a considerable number of published errors.
  • Topics in High-Performance Messaging
    Successful deployment of a messaging system requires background information that is not easily available; most of what we know, we had to learn in the school of hard knocks. To save others a knock or two, we have collected here the essential background information and commentary on some of the issues involved in successful deployments.
  • GD-Wheel: A Cost-Aware Replacement Policy for Key-Value Stores
    Currently , these key-value stores use either LRU or an LRU approximation as the replacement policy for choosing a key-value pair to be evicted from the store. However, if the cost of recomputing cached values varies a lot, like in the RUBiS and TPC-W benchmarks, then none of these replacement policies are the best choice. Instead, it can be advantageous to take the cost of recomputation into consideration.
  • Apache Kafka, Purgatory, and Hierarchical Timing Wheels
    Apache Kafka has a data structure called the "request purgatory". The purgatory holds any request that hasn't yet met its criteria to succeed but also hasn't yet resulted in an error. The problem is “How can we efficiently keep track of tens of thousands of requests that are being asynchronously satisfied by other activity in the cluster?”
  • SoK: Eternal War in Memory
    We systematize the current knowledge about various protection techniques by setting up a general model for memory corruption attacks. Using this model we show what policies can stop which attacks. The model identifies weaknesses of currently deployed techniques, as well as other proposed protections enforcing stricter policies.
  • Optimizing Hash-Array Mapped Tries for Fast and Lean Immutable JVM Collections
    In this paper we reduce memory overhead and runtime performance overhead from the implementations of immutable collections on the Java Virtual Machine (JVM).
  • What's Worked in Computer Science
    In 1999, Butler Lampson gave a talk about the past and future of “computer systems research”. Here are his opinions from 1999 on “what worked”.
  • Good Leaders are game changers: Raft & Paxos
    In this blog post, we will briefly show the similarities and differences between Paxos and Raft. Firstly, we will describe what a consensus algorithm is. Secondly, we will describe how to build a replication solution using instances of a consensus algorithm. Then we will describe how leaders are elected in both algorithms and some safety and liveness properties.
  • The story of one latency spike
    A customer reported an unusual problem with our CloudFlare CDN: our servers were responding to some HTTP requests slowly. Extremely slowly. 30 seconds slowly. This happened very rarely and wasn't easily reproducible. To make things worse all our usual monitoring hadn't caught the problem. At the application layer everything was fine: our NGINX servers were not reporting any long running requests.
  • Daily Report: Google Gets Serious About Competing in Cloud With Amazon Web Services
    One of the great mysteries of the tech industry in recent years has been the seeming disinterest of Google, which is now called Alphabet, in competing with Amazon Web Services for corporate customers.
  • A Cabinet of Infocom Curiousities
    If you’re coming into this relatively new, or even if you need a little brush-up, let me state: Steve Meretzky has earned the title of “Game God” several times over, having been at the center of the early zenith of computer games in the 1980s and persisting, even thriving, in the years since. He continues to work in the industry, still doing game design, 35 years since he started out as a tester at what would become Infocom.

    But more than that – besides writing a large amount of game classics in the Interactive Fiction realm, he also was an incredibly good historian and archivist, saving everything.

  • It Was Never Going to Work, So Let’s Have Some Tea
    "The master's tools will never destroy the master's house"
  • Rainbow Six Siege Review: This Thing Is Disturbingly Real
    Ubisoft’s latest tactical shooter, Tom Clancy’s Rainbow Six Siege, adopts a striking bent towards a unique brand of pseudo-realism. Siege evokes a perverse version of the uncanny valley. It mixes the over-the-top, arcade-style renditions of violence games often lean towards with the gut-wrenching reality that we are, in fact, remarkably fragile.
  • How Fallout 4 Mastermind Todd Howard Builds His Epic Dream Worlds
    “I think if someone has a gaming obsession, Ultima became mine,” says Howard. “I would say no other series ingrained itself in how I want to make games or what I want them to be more than Ultima did.”
  • Why Ball Tracking Works for Tennis and Cricket but Not Soccer or Basketball
    Most ball tracking systems rely on two different approaches. The first looks to follow the movement of the ball in three dimensions and then predicts various likely trajectories in the future. This “tree” of possible trajectories can then be pruned as more ball-tracking data becomes available.

    The advantage of this approach is that the laws of physics are built in to the trajectory predictions so unphysical solutions can be avoided. However, it is hugely sensitive to the quality of the ball tracking data and so tends to fail when the ball is occluded or when players interact with the ball in unpredictable ways.

    Another method is to track the players and note when they are in possession of the ball. The movement of the ball is then assumed to follow the player and when possession transfers from one player to another. The advantage here is that the system does not get so confused by rapid or unpredictable passes—indeed, this approach works well in basketball, where dribbling and occlusion can make life difficult for ball trackers. However, without physics-based constraints on the motion of the ball, these systems can produce inaccurate tracks.

  • How To Make Millions Of Hoverboards (Almost) Overnight
    Shenzhen is also, and only very recently, the hoverboard manufacturing capital of the world. In the smoke and asphalt of Bao An, a sprawling industrial flatland roughly the size of Philadelphia that serves as one of the city’s main manufacturing districts, hundreds of factories churn out much of the world’s supply of the boards, which are then shipped, rebranded, and sold around the globe.

And a question for the weekend: how many hours have you spent on your Hoverboard? (And no, I don't mean these hoverboards.)

Thursday, December 3, 2015

It's not just a game, ...

Open-World Games Are Changing the Way We Play

It’s a slow cartography, maps of imaginary spaces growing in my head, inch by inch.

I feel like this might be the way well crafted open worlds are supposed to be experienced—not as gluttonous binges or narrowly focused rampages, but as long-term occupancies. I’ve found that these games exist more vividly in my mind as I embrace this style of gameplay. They grow in my imagination as they occupy more and more space in my memory. Instead of rushing through them or viewing them as content generators, I abide in them.

Monday, November 23, 2015

The Witcher 3 Hearts of Stone expansion: a very short review

It's rather remarkable that I'm motivated enough to write a review of an expansion pack for a video game, but then again, The Witcher 3 is no ordinary video game, and the Hearts of Stone expansion is no ordinary expansion.

I don't spend anywhere near as much time playing video games as I once did; still, Steam's tracker will confirm that nearly all of my spare time over the last six weeks has been devoted to the enthralling Hearts of Stone adventure.

It's almost as though the base game were just giving the CD Projekt Red team a chance to warm up, and Hearts of Stone was the true expression of their craft. The characters are fascinating; the story is absorbing; the music and graphics and setting are just as beautiful as you've come to expect with this game.

It all starts with a good villain, and Hearts of Stone has a superb one: "Evil Incarnate," as one minor character informs us in a heart-wrenching recollection of how his life was destroyed by this creature.

This villain is one side of a Faustian pact-with-the-devil plot involving ruined aristocrat Olgierd von Everec, who, in a desperate attempt to recover from a youthful mistake and save his threatened marriage, makes a deal whose consequences he surely failed to anticipate.

"Beware of immortality," Olgierd tells us, "it's not all it's cracked up to be."

So what is in this expansion?

  • Miles and miles of new territory to wander and new locations to explore
  • A giant death-dealing frog in the sewers
  • An auction (bring a full wallet!)
  • A rune master from a foreign land
  • Shani, medic extraordinaire, with a complex story of her own
  • A bank robbery, in which you have to assemble a team and execute your plan
  • The full-and-detailed exploration of the life and affairs of the above-mentioned Olgierd von Everec, with long and crucial detours into the history of his wife Iris and his brother Vlodomir

And more, much much more.

My absolute favorite part, though, and probably the best-executed part of any video game I've ever played, is the stupendously wonderful wedding scene. Our hero (the witcher Geralt of Rivia) is rather a straight-laced sort who generally plays things quite close to the chest and doesn't let his guard down.

But as the story plays out, Geralt finds himself accompanying Shani to a friend's wedding.

However, Geralt has been "possessed" by the spirit of a rather carefree aristocrat, rather a rake in fact, who takes a completely different approach to attending this courtly event of high society, and the result is glorious! It's no lie to say that the events of the wedding found me laughing out loud at my computer, over and over again, as our hero finds himself (mis-)behaving in the most amusing ways.

It will be VERY hard to top Hearts of Stone, and somehow I think that, for some time to come, other games are going to seem drab and ordinary after playing The Witcher 3.

Happily, there is still next spring's Blood and Wine expansion to look forward to; what will those creative folk at CD Projekt Red think of next?

Saturday, November 14, 2015

It looked like nothing, and that was on purpose

A nifty short article on Wired about the demolition of E3: Watch Part of the Old Bay Bridge Implode.

Just after a lovely sunrise, a series of muffled booms shot across the San Francisco Bay, and a plume of water swallowed a huge chunk of the old Bay Bridge.

The chunk was once the E3 pier, a 20 million-ton concrete strut that reached from the water’s surface to the a foundation 50 feet below. At 7:18 am this morning 60,000 pounds of dynamite crumbled it into a hollow cylinder encased in Bay mud.

As the article goes on to explain, this was (deliberately) a very tame demolition from a spectator point of view. I was quite interested in this part:

Those two barges were a key part of the joy-killing efforts to conserve local wildlife. A series of hoses deployed from each sprayed the underwater portion of the pier in bubbles. "The bubble curtain is to contain the shock wave from the implosion," said Leah Robinson-Leach, CalTrans’ spokesperson for all things San Francisco Bay Bridge. To further spoil the fun (or protect people and structures safe from flying debris, again, depending on your perspective), the 80-by-140 foot rectangular top of the pier was covered by a huge steel and wood mat.

"Bubble curtain"? COOL!

I wonder if there is any underwater video of that part?

It is super-important to keep the San Francisco Bay as clean as possible, which is a real challenge given all the uses of the Bay.

So I'm pleased to see Cal Trans do what they can to avoid making matters worse, whenever possible.

And it was a cool little video in the article -- check it out!

Thursday, November 12, 2015

In which people discuss things I don't understand

The first link is REALLY worth a read ... very eye-opening and intriguing.

  • Uber's Drivers: Information Asymmetries and Control in Dynamic Work
    The Uber driver workplace is characterized by constant change and by remote management structures, such as algorithms, Community Support Representatives, and passengers, removes the governing responsibility for a reliable workplace away from a central actor – Uber as a corporate entity, or a singular managerial body. Drivers must compare the information they gather from their own experiences with CSRs, media reports, company statements, written policies, notices from local markets, and their own advice in forums as though there is a singular, sense-making machine at work. There are multiple authorities for what Uber says or does that drivers rely on because the Uber system provides the architecture for digital and physical points of engagement and interaction with different authoritative actors. As a case study in the emerging on-demand economy, our analysis of the Uber driver experience signals the need for further study of the social and technical dynamics of distributed work systems.
  • The Guilded Age
    Uber’s regulatory battles will, to some extent, pave the way for other services, be they car-hailing apps or delivery networks or privatized replacements for public transit or just other types of on-demand labor whatevers. Airbnb’s will free up, to some extent, Airbnb competitors. But because they’re first, and because they’re huge, and because their investors have lots of adjacent interests, these regulatory battles belong to them. This means our next laws regarding how people drive and get driven, and the next sets of rules determining what and where a hotel can be, will be written largely by these companies.
  • Airbnb Is Building An Army
    Airbnb policy staffers are already on the ground around the world. Lehane said in cities where clubs are founded, staffers will be called on to organize training, facilitate resources, and otherwise manage the beginnings of an international grassroots network. While most cities don’t have the same proposition system as San Francisco, which allows voters to weigh in directly on initiatives, he said he could foresee clubs supporting political candidates who are in favor of short-term renting and home-sharing in their cities. Lehane compared the potential political strength of Airbnb hosts and guests, of which there are over 4 million in the United States, to that of the National Rifle Association or the Sierra Club.
  • Peeking Beneath the Hood of Uber
    In order to understand the impact of surge pricing on passengers and drivers, we present the first in-depth investigation of Uber. We gather four weeks of data from Uber by emulating 43 copies of the Uber smartphone app and distributing them in a grid throughout downtown San Francisco (SF) and midtown Manhattan. By carefully calibrating the GPS coordinates reported by each emulated app, we are able to collect high-fidelity data about surge multipliers, estimated wait times (EWTs), car supply, and passenger demand for all types of Ubers (e.g., UberX, UberBLACK, etc.).

Wednesday, November 11, 2015

Quite a bit more than the whole 9 yards...

In the annals of construction porn, this is an oddly mesmerizing little gem: an 8.5 minute condensed time-lapse video of the 16 hour foundation pour of the new super-skyscraper that's now underway in the heart of San Francisco.

Wired reports with more details: It Took 18 Hours to Pour San Francisco's Biggest-Ever Concrete Foundation

All that concrete did not slop down into an earthy void. In the weeks leading up to the pour, workers constructed a subterranean lattice of rebar—12 layers high, with six inches separating each layer. "We used 5 million pounds of number 18 rebar, the largest size available," says Tymoff. At two and a quarter inches in diameter, grabbing a bar of No. 18 is like gripping the fat end of a baseball bat. It took eight iron workers to lift each 45-foot segment into place. That cagework will act as the foundation’s skeleton, but during the pour it also served as a catwalk for workers holding the cement hoses or the massive vibrators used to ensure the concrete had no air pockets.

The hardened slab, in all its mightiness, is but half of the tower’s earthquake protection. It will keep the building from toppling sideways, but what about sliding back and forth? In a big earthquake, the ground is actually trying to slip sideways underneath the building. "You need something to keep you from changing addresses," says Joseph. Those somethings are called piles, in essence underground stilts connecting the building with the bedrock. In the lowlands of San Francisco’s Financial District, bedrock is 300 below street level. "We have 42 piles that go all the way down and are socketed 15 to 25 foot deep into the rock," says Tymoff.

And of course, for even MORE detail, don't miss the wonderful site run by architects Pelli Clarke Pelli: Salesforce Tower.

At its base, Salesforce Tower connects directly to the transit center, which will house 11 Bay Area transit systems. On top of the Transit Center and linked directly to the tower is a 5.4-acre public park, which will offer recreational, educational, and nature activities. The park has two roles: the future anchor of the neighborhood and a key element of the project’s sustainable design strategy.

Each floor of the tower will have integrated metal sunshades, calibrated to maximize light and views while reducing solar gain. High performance, low-​emissivity glass will also help to reduce the building’s cooling load. Cooling may be provided in part by heat-​exchanging coils wrapped around the tower’s foundations. The tower and transit center also include comprehensive water recycling systems. In addition, high efficiency air-​handlers will take in fresh air on every floor.

Or, if you just can't stand it, head on over to the skyscraper's own website run by Boston Properties, and keep up with the minute-by-minute progress on Construction Cam!

It's interesting how these giant construction projects go. A few years back, I was completely fascinated by the new Bay Bridge, and in particular by the custom barge-based floating crane that was commissioned and delivered especially for the project: the Left Coast Lifter.

Now the bridge is built (and in fact the old bridge is pretty well completely torn down and removed), and the Left Coast Lifter hasn't been around these parts for years.

But last week, when I was in New York, we happened to make a side trip (to Storm King -- that reminds me, I need to blog about that, too!), and we found ourselves crossing the Tappan Zee Bridge.

And there, what did I see to my delighted eyes?

It's the Left Coast Lifter!

Alive and well, it's happily sitting in the Hudson River in upstate New York, contentedly building the new bridge.

The folks on that side of the country call it the I Lift NY supercrane.

But as we whizzed by on the super-speedway I could still make out the words painted across the bow:

Left Coast Lifter

So there you go.

Tuesday, November 10, 2015

New York City, Fall 2015

It came to pass that we had the opportunity to spend 48 hours in Manhattan, wandering around and enjoying ourselves.

And so we did.

Manhattan is so big and complex that it would take months, years, perhaps your entire life, in order to really understand it.

But we didn't have that; we just had 48 hours.

So we had to concentrate, and pick a few things.

It so happened that we arrived in New York fairly late in the afternoon. By the time we had checked in to our hotel, it was already dinner time, so we went down to a nice (although quite busy) little restaurant in the East Village (just off St Mark's Place) for a nice meal.

After dinner we got back to the hotel, but we weren't quite ready to call it quits, so we went up to the 48th floor, where the revolving rooftop bar made a delightful location to have a drink before bed. It's a great experience; a fun touch is that the cocktail napkins are printed with a circular "skyline map" identifying all the buildings that you see, so that as you rotate around you can make sense of what you're looking at.

Assuming you're brave enough to actually look out the window, that is, and aren't just clutching your table and chair as tightly as possible (really? did I do that?)

Originally, we were planning to take a boating cruise in the morning; there are several of them which circumnavigate Manhattan, and it seemed like a relaxing way to see a lot of New York City (from the water). But the cruise was cancelled and so we didn't go; in retrospect, this was probably to our benefit, as the weather that day was misty and with very low clouds, so much of the city would have been hidden in the haze.

Instead, we made our way down to Battery Park and Castle Clinton and took the ferry to Liberty Island and on to Ellis Island. Although the weather was indeed gray and misty, in a way this rather enhanced the trip, as Liberty Island emerged from the clouds to our great delight.

We didn't have the fancy tickets to climb up into the statue itself, so we contented ourselves with walking around the island and looking at the statue from ground level, which was quite enjoyable.

Then we returned to the ferry and proceeded on to Ellis Island. Although it doesn't have the Statue of Liberty on it, Ellis Island is in many ways a much more interesting place.

Over the last few decades, the Ellis Island facilities have been converted to an immense and extremely well-organized museum, telling the story of immigration and how it built the United States of America.

The main building is 3 massive stories tall, and nearly all of it is museum. Even though many of the exhibits are straightforward, and we made an effort to move through in a lively fashion, it was well over an hour to see what we saw, and I think we saw barely half of what there was to see.

The ferry returned us to Battery Park, and after some wandering around, and some lunch, the weather had cleared nicely, and we made our way up to Central Park.

I was very interested to see the John Lennon memorial in Central Park, and I wasn't disappointed. It is quite nice, and it was filled with people like myself, stopping to look and think a bit before moving on, all of us quietly part of a shared experience.

The weather was glorious, so we walked around Central Park for several hours. We moseyed across from Central Park West to Central Park East, stopping at places like the Bethesda Fountain, the Hans Christian Anderson statue, the Model Boat pond, and the Alice in Wonderland statue.

Every so often we would wander out of the park, but the surrounding areas weren't as nice, so we just kept wandering back into the park, walking up and down the tree-lined paths, marveling at all the different things to see.

After a while we were tired, so we found a nice spot on the Upper West Side to sit for a while and rest and talk; when we were restored it was already getting on to dusk, so we made our way down to Lincoln Center to see the fancy theaters.

Later we made our way out to Rockefeller Center, which was already all lit up for the holidays. We tarried for a while, watching the ice skaters in the ice rink, and wandering through the enormous Lego Store.

It was dinner time, and my plan had been to find one of the up-and-coming Indian restaurants in the so-called "Curry Hill" neighborhood near 28th and Lexington, but instead we ended up at a very nice spot a little bit farther north in Murray Hill, where we had a fine meal.

The next morning, we popped out of bed again and headed back downtown to the 9/11 Memorial. Although we were both well-acquainted with the events of 14 years ago, neither of us had been to Manhattan since, so we wanted to make a visit to the memorial part of our trip.

This is an extremely dramatic and moving place, obviously, and the memorial accomplishes its task(s) well, I thought. The overall presentation is quite remarkable: from the street-level entrance you make your way down, down, down. The farther you go, the more dramatic and powerful the experience becomes, until you reach the very bottom, where the bulk of the exhibits and memorial materials are located.

I was pleased to see that, for the most part, the memorial lets the facts speak for themselves, and focuses its attention on the people who were most directly affected: those in the towers, on the planes, and at the Pentagon, as well as the emergency personnel who responded to the events.

The displays are physical and immediate, incorporating objects from the buildings themselves (the stairs, the foundation columns, the slurry wall, the steel girders, etc.) as well as objects from the people involved (equipment, personal effects, etc.)

The memorial uses multi-media EXTREMELY effectively, playing actual clips from television broadcasts, 911 recordings, cell phone messages, interviews with witnesses and survivors, etc. A particularly dramatic and moving exhibit tells the remarkable (if by now quite well-known) story of Flight 93, moving back and forth between air traffic control recordings, voice mail messages, and other information to let the actual participants in the story tell it, speaking from the grave as it were in some cases. I glanced into that room for just a moment but was instantly captivated, and 10 minutes passed before I could breathe.

We hadn't expected to spend long at the memorial, but before we knew it we'd been there more than 2 hours, and had to drag ourselves away and on. Although upon leaving I felt like I hadn't really learned anything I didn't already know, I still felt like my visit was valuable and I don't regret going for an instant.

We both really enjoyed wandering around the various Manhattan neighborhoods, and I think we could have done much more of this if we'd had time. Some are rather straightforward, like walking through the Financial District or down the canyons of skyscrapers mid-town.

Others are still full of personality and character, like Greenwich Village, SoHo, the East Village, Murray Hill, or Chelsea. We got just ridiculously lost wandering around Greenwich Village: one of my personal goals had been to visit the Village Vanguard, but we ended up abandoning that quest and moving on; later, looking on the map, I realized that we had stopped in a falafel shop that was, quite literally, across the street from the Village Vanguard, and hadn't even known it.

New York is definitely quite expensive, and eating and drinking there was not cheap by any stretch. However, the food was remarkably good, much better than either of us had expected, full of fresh and good ingredients, well-prepared, well-presented, well-delivered. If I could afford it, I could easily spend all my time just wandering around Manhattan, eating and drinking and looking about...

Another very nice surprise was how successful we were at taking the subway all over the island, even with very little advance preparation and zero experience with the things that it often turns out you need to know about a city's transit system.

The hardest part of using the subway turned out to be finding the stations from above-ground. Once you were in the station, though, everything was well-marked and easy to find and in remarkably good condition given the astonishingly heavy use that the New York City subways receive.

Trains ran regularly, loudspeaker announcements and display signs were clear and accurate, the system as a whole seemed to be basically clean and safe, and all in all it was much better than I had anticipated.

That said, the subway was certainly not as nice as Seoul's subway, which is perhaps no surprise because Seoul's subway is brand new by comparison. As compared to the London Underground, though, I thought that the New York City subway system was at least as good, and certainly not as complex.

If you find yourself in La Guardia Airport, and want to get into midtown, the NYC Airporter is just fine, and certainly a bargain compared to airport-to-downtown options I've used elsewhere.

If you want a place to stay, and are looking to be part of everything, the Marriott Marquis is right smack in Times Square, in the middle of the action, but once you duck into your room and close the door, it's peaceful and welcoming as can be. And boy is it convenient to get to anywhere else in Manhattan from that location!

Oh, and my wife got to go see Kinky Boots at the Hirschfeld on Saturday night, but you'll have to ask her about that (I was in Port Chester at the time, as we've already discussed).

Monday, November 9, 2015

Phil Lesh and Friends, Capitol Theatre, Port Chester, NY, Nov 6 & 7, 2015

Nowadays, in the world of music entertainment, the typical band has a stable and well-known roster of performers; those performers are the primary reason you choose to attend one performance versus another, after all.

And the typical tour has a stable and predictable program, drawn from an obvious repertoire; the most common justification for a tour nowadays is to promote a new album release, and hence it's almost universal to expect that the performance will consist primarily of material from that new release.

Phil Lesh is typical in neither way.

Lesh, one of the founders and the former bass player for the Grateful Dead, has evolved a most interesting and unusual performance style which he calls "Phil Lesh and Friends".

After more than 50 years as a professional musician, Lesh has an enormous number of musical contacts, as well as an extensive and diverse collection of material.

There are even those who give him credit for how the band got its name:

What matters for our purposes is that Wenner, arguably the 20th century’s most important and influential rock journalist and publisher, got his scoop on the band’s name directly from the its bassist, Phil Lesh, who played an important role in giving the band its name—it was at Lesh’s home that Jerry Garcia came upon the phrase "The Grateful Dead" in "a big Oxford Dictionary," as Garcia remembers it in Signpost. That may be why the name was so fresh in Lesh’s mind when he told Wenner "We’re the Grateful Dead."”

And, perhaps most importantly, he has wide-ranging interests and a genuine joy of performance, which drive him to find ways to continue playing and interacting with his audience.

So, his format (roughly) is this: every so often, Lesh contacts some number of his friends, who clear time on their schedule, and make some arrangements to meet and discuss and prepare.

Then, at the appointed date, and at the appointed location, Phil Lesh and Friends appear, and deliver their show.

It's never the same show twice.

You never know ahead of time who's going to be in the band (except, of course, for Lesh).

And you never know ahead of time what will be on the program (although, broadly speaking, you know what sort of material it will be, since after 50 years everybody knows what sort of music Lesh enjoys).

It's a pretty unusual format. And, given that Lesh is now 75 years old, and has had numerous health problems (liver transplant, prostate cancer, bladder cancer), you never know how much longer you might get a chance to see him in action.

And so it came to be that, mostly as an excuse for a far-too-long-postponed visit to my very oldest and dearest friends on the planet, we hopped on the plane and I came to be in Port Chester, New York, on November 6th and 7th, 2015.

It's worth, as a side-note, mentioning why, specifically, we were in Port Chester. Although the Capitol Theatre was famous, 45 years ago, as the site of some of The Grateful Dead's most famous shows, the theater had become disused and was idle until recently. However, as part of its re-opening, Lesh was named "musician in residence" and has been playing there regularly, and it is clearly one of his favorite places to play.

It's hard to over-state the difference between seeing a show at the Capitol Theatre versus almost any other that shows acts of this caliber. The theater holds fewer than 2,000 people; in contrast, when I saw The Grateful Dead in June here in California, there were nearly 50 times as many people in the audience, and we spent the time watching the show on 70-foot-tall video screens, for the most part.

But at the Capitol, the space is small and friendly and personal. You can almost imagine that you have been invited into their living room and you are sitting on the couch, listening to them play and sing and talk and relax.

Well, you and your 1,799 new best friends, that is.

So, on to the music.

During this particular event, Phil Lesh's friends were David Nelson, Barry Sless, Scott Law, Jason Crosby, and John Molo.

Of those musicians, David Nelson is probably the most famous. He might be best known for his group New Riders of the Purple Sage, but he also played with the Grateful Dead many a time back in the day; for instance, he plays the electric guitar on the recording of Jack Straw on American Beauty. As the group's web site recalls:

In the summer of 1969, John Dawson was looking to showcase his songs while Jerry Garcia was looking to practic his brand new pedal steel guitar. The two played in coffeehouses and small clubs initially, and the music they made became the nucleus for a band - the New Riders of the Purple Sage.

That same year, David Nelson, expert in both country and rock guitar, joined the group on electric lead guitar.

As you might expect from this line-up, the selection of music for the two shows included several New Riders songs: John Hardy's Wedding, Garden of Eden, Bob Dylan's The Wicked Messenger, and of course their signature song, The Adventures of Panama Red.

And naturally there were a broad range of Grateful Dead signature songs, including several of Lesh's own compositions: Box of Rain (which is a personal favorite of mine), Pride of Cucamonga, Mason's Children (which disappeared from the Dead's regular rotation long before I started following them closely), and Unbroken Chain, as well as Grateful Dead classics not so closely linked with Lesh, such as Jack Straw (led, as a delightful surprise, by Lesh's son Grahame Lesh on vocals and guitar), Uncle John's Band, Dire Wolf, Cold Rain and Snow, and Scarlet Begonias

Perhaps because of the musicians that were particularly present for these concerts, the music selection also drew from the Grateful Dead's long history of blues, bluegrass, boogie-woogie, and jug band traditions, including pieces such as Goin' Down the Road Feeling Bad, White Lightning, Loose Lucy, Turn On Your Love Light, and Not Fade Away

But most interesting of all, musically, I think, was the inclusion of three fairly unusual songs from an American musician who is not so well known at all, I think: Noah Lewis. Let's let's biography of Lewis tell some of his story:

A key figure on the Memphis jug band circuit of the 1920s, singer and harpist Noah Lewis was born on September 3 of either 1890 or 1895 (depending on sources) in Henning, Tennessee. Upon relocating to Memphis, he teamed with Gus Cannon, becoming an essential component of Cannon's Jug Stompers; the group made their debut recordings for the Paramount label in 1927, with several more sessions to follow prior to their final date in late 1930. On a series of sides cut in the first week of October 1929, Lewis made his debut as a name artist, cutting three blistering harmonica solos as well as "Going to Germany," which spotlighted his plaintive vocal style. Later recording with Yank Rachell and John Estes, as the Depression wore on Lewis slipped into obscurity, living a life of extreme poverty; his death on February 7, 1961 was a result of gangrene brought on by frostbite.

Lewis died before I was even born; this is OLD music as far as things go in American music history.

What is the "Jug Band"? Well, again, let's turn to's site for more information:

Jug bands united Appalachian folk with blues, ragtime, and very early jazz; they are best known, of course, for their novel, do-it-yourself instrumentation. The jug in question was usually a whiskey jug, and a player blew across the mouth of the jug to produce pitches in the bass register. Jug bands usually featured at least one stringed instrument from the Appalachian tradition -- guitar, banjo, and/or fiddle -- and used a wide variety of everyday, easily available household objects for rhythmic accompaniment. The most common were the washboard (whose slats were struck and rubbed in a way analogous to a snare drum) and the metal washtub bass, which was usually equipped with a broomstick and clothesline that produced the sounds. Other possible percussion instruments included spoons, gut buckets, bones, and saw blades; additional melodic accompaniment might have included a harmonica, kazoo, or even comb and tissue paper -- whatever was available and economical, really. Jug band music originated in Louisville, Kentucky at the dawn of the 1900s, but found its greatest popularity in Memphis, Tennessee during the '10s and '20s, eventually spreading to Ohio and North Carolina as well. Given the inherent playfulness of the instrumentation, jug band music was accordingly informal, spontaneous, often humorous, and rhythmically bouncy.

Jug Band music and The Grateful Dead have gone together for at least 50 years, of course, but it was quite pleasing to me to see the selection of three of Noah Lewis's pieces in Saturday's program: Minglewood Blues, Big Railroad Blues, and Viola Lee Blues.

The three songs have some interesting differences. Big Railroad Blues is a crowd-pleasing sing-along, a blues song with a light-hearted sense of irony and and humor, as our hero sings:

Well my mama told me, my papa told me too,
Now my mama told me, papa told me too,
Well I shouldn't be here tryin' to sing these railroad blues.

Wish I had a'listened to what my mama said,
Wish I had a'listened to what my mama said,
Well I wouldn't be here tryin' to sleep in this cold iron bed.

Minglewood Blues, on the other hand, as my co-worker "reb" notes, has a long and complex history. What or where is "Minglewood"? Well, The Old Weird America takes a stab at clarifying it:

The answer to where exactly is this "Minglewood" is a bit uncertain. I have read somewhere that it was a lumber camp near the Mississippi where musicians (including Noah lewis and Gus cannon) gathered on weekends to have a good time, and judging from the lyrics of the "New Minglewood Blues" that Noah Lewis recorded with his own jug band ("If you’re ever in Memphis, better stop by Minglewood"), it was a place in the city or close to it.
Make sure you follow the link from that site to The Myth of Minglewood for more backstory about Minglewood.

But, for my tastes, the best of the three Noah Lewis songs was the third one, the heart-breaking and beautiful Viola Lee Blues. This song meant so much to Lesh and the rest of The Grateful Dead that they made it the 10 minute long epic climax of their first actual album.

The wonderful Grateful Dead Guide site discusses the 50 year history of The Grateful Dead's history with the song, and how it morphed and evolved through the years; it's clear that it meant a tremendous amount to them.

Viola Lee Blues was the Dead's first big jamming tune. Dating from the start of their career when they were doing mostly pop and blues songs, they designed it as a psychedelic trip: it would start as a strange old jugband tune with dark chords, a constricted groove, and wailing black-harmony vocals, but the music in-between the verses would gradually stretch out to unreasonable lengths and start accelerating until the band were playing fast, shrieking gusts of sound, tearing open the fabric of reality -- then suddenly the noise stops and the song jauntily reappears again. As one writer has said, it may have been a one-dimensional song, but that happened to be the fifth dimension!

You can find many of The Grateful Dead renditions on the net, of course, but here's an alternate suggestion instead: listen to this wonderful performance by Jim Kweskin's Jug Band while you follow on with the lyrics here, since they're a bit hard to make out until you've heard the song a few hundred times:

The judge decreed it, the clerk he wrote it.
Clerk he wrote it down indeed-e
Judge decreed it, clerk he wrote it down
Give you this jail sentence you'll be Nashville bound

Some got six month some got one solid.
Some got one solid year indeed-e
Some got six month some got one solid.
But me and my buddies all got lifetime here

I wrote a letter I mailed in the air,
Mailed it on the air indeed-e
I wrote a letter I mailed in the air.
You may know by that I've got a friend somewhere

It's nearly a hundred years since Noah Lewis penned his sorrowful, tragic, heart-breaking tale of injustice, loneliness, and despair, but that amazing, glorious final verse, with its simple recognition that that simple act of writing a letter, of reaching out, of trying to communicate with some other human being somewhere else, has the power to overcome that cruelty and show the world that "I've got a friend somewhere."

This is

fast, shrieking gusts of sound, tearing open the fabric of reality

So even though there were lots of wonderful, wonderful things to remember about these shows, the sentiment of this majestic hundred-year-old song appealed to me on my madcap visit to my friends-of-four-decades, and somehow it seemed the perfect way for me to try to make sense of the entire experience.

Tuesday, November 3, 2015

Under the clock

I'm told that, nowadays, when people say "meet me under the clock," they mean the clock in Grand Central Terminal.

But, apparently, 70 years ago it meant the clock in the Astor Hotel: The Clock.

The Clock (UK title Under the Clock) is a 1945 American romantic drama film starring Judy Garland and Robert Walker and directed by Garland's future husband, Vincente Minnelli. This was Garland's first dramatic role, as well as her first starring vehicle in which she did not sing.

I'm always fascinated by how expressions like that evolve and change while still retaining that essential meaning.

Saturday, October 31, 2015

Hearts of Stone

The Witcher 3: Hearts of Stone review: The devil went down to Novigrad

In my mind I imagine a Witcher 3 without Ciri. One which embraces Geralt's wandering nature, his inclination towards short-term contract work, and relies on these ten-hour plots. A world the size of Novigrad/Velen/Skellige but filled with all manner of quests from one-off contracts to—on the other end—these heavily character-driven vignettes of a witcher's life.

Friday, October 30, 2015

Golden Joysticks 2015

Frankly, I had no idea that there was an annual awards ceremony called the Golden Joysticks.

Shows how much attention I've been paying.

But ... Golden Joysticks 2015: The Witcher 3: Wild Hunt wins five gaming awards

The action, role-playing game won ultimate game of the year, best storytelling, best visual design and best gaming moment.

"Best gaming moment"?

Oh, I see:

Best gaming moment: Bloody Baron quest in The Witcher 3: Wild Hunt

Yes, that definitely deserved an award.

Sunday, October 25, 2015

Things to read, mid-October edition

We took a short trip to the mountains, to see if it was as dry as the reports all said.

It was even worse.

Still, the mountains are beautiful, even when they are parched.

Meanwhile, the reading list only grew while I was gone...

  • In Defense of The New York Times
    In other words, the job of The New York Times is no longer to produce "All the News That’s Fit to Print"; rather, it is to invest in stories that make a difference — stories that start a conversation — and trust that readers will be willing to pay for quality. The content follows from the business model.
  • '10-second' theoretical hack could jog Fitbits into malware-spreading mode
    The athletic-achievement-accumulating wearables are wide open on their Bluetooth ports, according to research by Fortinet. The attack is quick, and can spread to other computers to which an infected FitBit connects.
  • The NSA and Weak-DH
    They further observed that most servers using this for IPsec, a major Virtual Private Network protocol that encrypts a large amount of business traffic, commonly use the same p and g, and most of these systems are using 1024b Diffie-Hellman.

    So with an NSA-style budget of a few hundred million dollars, one could build a supercomputer that can first perform a huge amount of work, running for months, in order to break a particular 1024b p and g and then, using the same supercomputer, quickly break any key exchange using that particular p and g. This wouldn’t work for longer keys (such as 3072b Diffie-Hellman), elliptic curve Diffie-Hellman, or RSA encryption.

  • I'm Shocked, Shocked to Find There's Cryptanalysis Going On Here (Your plaintext, sir.)
    There's also been unhappiness that IPsec uses a small set of Diffie-Hellman moduli. Back when the IETF standardized those groups, we understood that this was a risk. It's long been known that the discrete log problem is "brittle": you put in a lot of work up front, and you can solve each instance relatively cheaply. The alternative seemed dangerous. The way Diffie-Hellman key exchange works, both parties need to have the same modulus and generator. The modulus has to be prime, and should be of the form 2q+1, where q is also a prime. Where does the modulus come from? Presumably, one party has to pick it. The other party then has to verify its properties; the protocol has to guard against downgrades or other mischief just in agreeing on the modulus. Yes, it probably could have been done. Our judgment was that the risks weren't worth it. The real problem is that neither vendors nor the IETF abandoned the 1024-bit group. RFC 4307, issued ten years ago, warned that the 1024-bit group was likley to be deprecated and that the 2048-bit group was likley to be required in some future document.
  • Fun with recreating an evil merge
    A good news is that, when the evil merge is in a file that also has textual conflicts to resolve, "git rerere" will automatically take care of this situation. All you need to do is to set the configuration rerere.enabled to true before attempting the merge between X and B and recording their merge M, and then attempt a new merge between B and Y. Without even having to type "git rerere", the mechanism is invoked by "git merge" to replay the recorded resolution (which is where the name of the machinery "rerere" comes from). A bad news is that when an evil merge has to be made to a file that is not involved in any textual conflict (i.e. imagine the case where we didn't have "line added by A" vs "line added by X" conflict earlier in the same file in the above example), "rerere" does not even kick in. The question is what to do, knowing B, X, and M, to recreate N while keeping the adjustment needed for semantic conflicts to record M.
  • Storage Technology Roadmaps
    At the recent Library of Congress Storage Architecture workshop, Robert Fontana of IBM gave an excellent overview of the roadmaps for tape, disk, optical and NAND flash (PDF) storage technologies in terms of bit density and thus media capacity. His slides are well worth studying, but here are his highlights for each technology
  • Stacking Up The Next Modern Platform
    When we were building Google Compute Engine, I viewed virtual machines on GCE as a transitional technology. VMs will always be around but there is a better world out there if we get past the local maxima that we are at with the current infrastructure offerings.
  • ARM Server Market
    Clearly Intel still makes the CPU behind more than 90% of the world’s servers (even when taking a very generous interpretation of server). And, just as clear, Intel is a very competent company that has in the past responded quickly to competitive pressure. Intel has also gotten very good at working closely with its major customers and, unlike the bad old days, is actually very good to work with. I’m more impressed with what they have been bringing to market than ever. Nonetheless, there are factors that make it very likely that we are going to see some very good server parts based upon ARM in market in the near future. It’s hard to predict the pace of execution of any of the participants nor where this will end up but, generally, change and competition is good for the industry and great for customers.
  • It's happening - OpenSSH for Windows...from Microsoft
    Sure, it's late, and ya, it should have happened years ago, but it's happening and it'll be built in. SSH will be one less thing to worry about.
  • Overcomplexifying, Underdelivering
    While it is hard to draw definitive lessons from a handful of programs, it is nearly certain that IT modernization efforts will overrun their cost estimates by significant amounts. The chart also demonstrates the challenges of holding these projects accountable when they do: cost overruns, delays, and reduced functionality are so common that even self-proclaimed success-stories have them.

    One solution is to try to make more realistic initial estimates. More data is required (and if you know of a project we're missing, please leave a comment), but trying to combine more than 50 nontrivial legacy government IT systems for less than $400 million to $500 million seems nearly impossible. So we should be skeptical when we see overoptimistic—or outright fraudulent—project estimates that claim to do just that.

  • The Tide Turns on Big Outsourcing – on cloud, agile, and rebuilding skills
    The idea that massive outsourcing contracts don’t suffer from scope creep and related, massive cost overruns, would be funny were it not for the fact that in the public sector at least, it’s our money, paid in taxes, being wasted. The UK government has wasted tens of billions of pounds on failed IT projects over the last 20 years or so, and one huge step forward under the last coalition government was a more sensible approach to citizen service provision.
  • Negative Gross Margins
    We have seen a tremendous number of high growth companies raising money this year with negative gross margins. Which means they sell something for less than it costs them to make it.

    It can be an "on-demand" service provider that subsidizes the cost of the workers on its platform so that the service seems like it costs less than it actually does. Why would an on-demand startup take this approach? To build demand for the service, of course. The idea is get users hooked on a home cleaning service, a ridesharing service, a food delivery service, or a gym roaming service by bringing it to market at a price point that is highly attractive and then, once the users are truly hooked, take the price up.

  • A Pulitzer is no guarantee
    There is a problem with the article. It correctly credits the Internet Archive with its major contribution to Web archiving, and analogizes it to the Library of Alexandria. But it fails to mention any of the other Web archives and, unlike Jill Lepore's New Yorker "Cobweb" article, doesn't draw the lesson from the analogy. Because the Library of Alexandria was by far the largest repository of knowledge in its time, its destruction was a catastrophe. The Internet Archive is by far the largest Web archive, but it is uncomfortably close to several major faults. And backing it up seems to be infeasible.
  • The Little-Known Story Behind Britain's Road Signs
    Kinneir and Calvert created rules for traffic signs that have endured to this day. Consider the wide gaps in letter spacing typically seen on roadside signs: That spacing is derived from research the designers conducted on how type should scale according to the speed of traffic and the amount of information on display. For Transport, the unit of measure for spacing is based on the width of the capital letter ‘I’—a consistency in form which, over time, helped foster a sense of familiarity in drivers.
  • Best Haka Ever
    The dueling Sipi Tau and haka prior to the first-round match between Tonga and New Zealand at Newcastle’s St James Park was the most scintillating, intense, and beautiful performance of the dance in modern rugby history.

    Not only was it breathtaking, the dueling dances grounded the extraordinarily physical tone of the 80 minutes of rugby to follow. “We’re going to tell the whole world that God and Tonga is our inheritance,” Tongan center Siale Piatau explained before Tonga took the turf.

    It was impressive to see the heart with which Tonga—a tiny and impoverished nation—performed the Sipi Tau dressed in their traditional luminous red jerseys that contrasted sharply with New Zealand’s black. The All Blacks response was incendiary, 23 men moving with a razor-sharp unity and collective purpose. The resulting rugby was worthy of the display. Tonga’s superb first half was probably the best they’ve ever played. Meanwhile, the All Blacks, rebounded from an unconvincing victory over surprise darlings Georgia to claim the win, overcoming the Tongans with graft, flair, and legs, scoring seven tries.

  • How Prison Architect Could Liberate Gaming
    After being available for three years as an open, prerelease “alpha,” Prison Architect was officially released two weeks ago and appears destined for long-term cult success. With a current user base of more than 1 million players, many of whom have already been playing for months if not years, the release carried significantly less risk for both players and creators than most project launches, its slow launch limiting its vulnerability to the caprices of the market and the media. I spoke to Introversion Software’s Mark Morris about the company’s approach to crowdfunding, project management, and community relations.