Fifteen seconds remaining, down by one....Who you gonna foul?

I was following the Mavericks - Kings game earlier this year. Exciting game which went into overtime. With 50 seconds left and the Kings down by two, Rajon Rondo fouled Jason Thompson and in the process sent him to the free throw line. He made the first shot. But before he could attempt the second, the Mavs called a timeout. After the timeout, Jason went back to shoot his second free throw, and missed it.

Did the timeout have any impact on the missed shot? You couldn't make a free throw any more straightforward. Unlike a penalty kick at soccer, there's nothing your opponents can do to alter that shot. Except call a timeout? Legendary coach Phil Jackson was (in)famous for calling timeouts between opponents free throws, but was this ploy effective at all? Thompson could have tied the game on his second free throw attempt, which would have shifted a considerable amount of pressure of his and his teammates' shoulders with less than a minute to go. With the timeout called, Thompson was left there brewing in these thoughts with mounting pressure.

That game made me want to investigate the timeout phenomenon, as well as other external factors that could influence the outcome of a free throw. I also wanted to follow up on an earlier post I made about measuring players' clutch performances via statistical models.

A few words on the data before jumping into the analysis. I focused on the most recent complete NBA season: 2013-2014. I pulled all the play-by-play data from nba.com, and pulled free throw season percentages for each player from espn.
Quite a bit of cleaning up was required, namely around players with same last name and same team the worst example being the Morris twins in Phoenix who also share same initial!
After cleaning everything up, I was left with just under 56K free throws taken in that season, ready to be analyzed!

I was primarily interested in the impact of free throws interrupted by timeouts, but also wanted to capture two additional factors: whether the shooter has homecourt advantage or not, and whether the situation is "clutch". There are countless definitions of "clutch time" available, some sparking heated debates. I have here defined it as "less than 2 minutes to play in the 4th quarter or in overtime, and less than 5 point differential between the teams' scores".

Before jumping into the data and analysis, let's first do some visual explorations.

How many free throws are taken by quarter?

Not surprisingly, significantly more free throws are taken in the fourth quarter than the first. The game is on the line, the defense goes up a notch, and voluntary fouls are committed to regain ball possession and prevent the opponent from running down the clock.

We can even go down one granularity level at look at the number of free throws made by minute played. Rather impressive to visualize the steady increase throughout each quarter, and the giant spike in the final minute of regulation with teams fouling on purpose in tight games.

We've looked at volume, let's know look at efficiency. How well do the home and road teams shoot the ball?

It appears that both teams shoot at very similar rates throughout the contest, with the home team always having an advantage although it is not a significant as one might have expected given the distractions often displayed by the home fans.

Both teams seem to do better in overtime, but we need to caution against the much smaller sample size there.

And now to the more interesting piece, how do teams execute in clutch time?

Quite surprisingly, the home team appears to be performing no differently, whereas the road team gets a nice boost of almost 5%. The fact that we observe a boost might seem counterintuitive for some: under pressure, with fatigue from close to 48 minutes of gameplay, wouldn't it be more difficult to concentrate and sink the shot? However, a reverse argument could be made that especially when games are close, or when a team expects the other one to intentionally foul, the coach might chose to place his best shooters on the floor. So teams aren't necessarily shooting better, just having better shooters take the shots. This however does not fully explain why the road team has a boost and not the home team.

The following graph shows the 1st quantile, median and third quantile for season free throw percentage of the players taking shots in and out of clutch. It is rather apparent that better shooters are on the floor in clutch moments.

Now that we have a better feel for the data, the analysis can begin. The data is extremely rich and offers multiple options from a statistical analysis point of view. We can leave the baseline free throw shooting percentages for each player be determined by the model fitting, or force these to be the players' season averages. But with different players taking a very different number of free throws within a season, and strong dependency in the success of a free throw for all those taken by the same player, a hierarchical structure emerges and a mixed effects model could make sense.

I actually played around with the three options just mentioned, and was satisfied at how close the numerical outputs were to each other.

The conclusions would indicate that:

• homecourt does have a positive effect on shooters' success, although the effect was only borderline significant
• calling a timeout before the second (or third) free throw had a negative but insignificant impact
• clutch time had a negative and significant impact

Regarding timeouts, the fact that the effect was not significant could be due to the low sample size of these events (84 cases in 2013-2014 out of 56K free throws taken), more coaches should test this strategy so I can tell them if it's effective or not!

As for clutch time, the conclusion seems to contradict the visual exploration where percentages were higher in clutch time. But recall that our explanation to this was that the coaches were putting better shooters on the court. The analysis would indicate that even if the best shooters are on the floor in the closing minutes, they are individually performing less well when the game is on the line than in the middle of the second quarter.

Now one might wonder if we could use the data to detect some of the leagues best clutch free throw shooters. Those cold-blooded killers who can step it up an extra notch when all eyes are on them. The Durants, James, Bryants...

I added some interaction terms for players with sufficient (20) in and out of clutch time free throws and see which ones had the potential to elevate their game. And the results are... no one! Out of the 26 players meeting my criteria, none could significantly increase their free throw percentage. This could again be due to small sample size, but even so most players had negative coefficients. While none had significant positive coefficients, two had significant negative coefficients: Chris Paul and Ramon Sessions.

So back to the post's title, if you're playing the Clippers, fifteen seconds to go and down by one, do you foul Chris Paul?

Clutch or not clutch, that is the stat question

There has always been debate on what clutch is, how it is measured, who performs best in these moments, the list goes on...
I'm definitely not going to settle the debate once and for all in one blog post, but wanted to share a few thoughts and ideas instead.

When we talk about clutch in the NBA, a few names immediately come to mind, Jordan, Bryant, Bird, Miller and Horry. Countless lists can be found with the simplest search, all as subjective as the next ("that was an incredible play in that game!").

But can we actually measure it, and rank players by it?

nba.com has a whole section dedicated to clutch stats on its website. A great first step but amidst all the numbers it's hard to compare players.

SBNation also tackled the issue, clustering players into recipients, creators and scorers (not mutually exclusive) during clutch time (Is Kobe Bryant Actually Clutch? Looking At The NBA's Best Performers In Crunch Time). The article stresses the importance of efficiency by placing all performances in perspective using possessions per 48 minutes on the x-axis. Here are the results for the 2010-2011 season:

Efficiency is a trademark at SB Nation, and Kobe is their primary scapegoat given this viewpoint:

SBNation's perspective is interesting and allows swift comparisons across players, but I feel that it lacks some rigor and robustness around these numbers. How large are the sample sizes? Are the effects significant? Which players are shouldering the most pressure and confronting it head-on? The author underlines these issues himself:

But all of that said ... how reliable are these numbers? There's a school of thought that firmly believes that "clutch" is in the eye of the beholder. They contend that as fans, we see things that may not actually be there. We see Kobe hit a step-back 20-footer and credit his clutch ability, when perhaps we simply should have attributed it to the fact that he's amazing at basketball (in the 1st or 4th quarter).
There are rigorous methods of testing for statistical significance. Rather than dive into those, however, a glance at some yearly efficiency trends can be just as telling.

I also came across this very nice post, Measuring Clutch Play in the NBA, on the Inpredictable blog which offers an interesting and elegant alternative. In a nutshell, the idea is to look at how each player's actions impacted his team's probability of winning the game, referred to as Win Probability Added WPA. Made shots, rebounds, steals increase your team's probability, while missed shots and turnovers hurt it. Some adjustments are required to clean up the cumulative WPA for each player (essentially comparing the impact of the same play under normal circumstances), but it does at the end provide an intuitive metric that makes sense and allows quick comparisons.

I do however have some slight concerns with this metric. The first is that, unless I misread, the metric is cumulative, so that players with more minutes in the clutch have more opportunities to modify their team's WPA. The second is best illustrated with a small example: with a few seconds remaining, if a player makes a two-point shot with his team down by 2 or down by 1, it will make a huge impact on the WPA: in the first case they're tied, likely to go to overtime with 50/50% for each team to win the game, in the other case his team leads by 1 and have a good chance of winning the game. But is it fair to credit the player with very different WPA in both cases? What really matters is that, under tremendous pressure, the player made the shot.

This in turn leads to another question: what was the likelihood of that shot going in in the first place? How frequently does that player make that shot under normal circumstances without the game on the line? How frequently do other players make the shot? How much does clutch pressure reduce the average player's chance of making the shot, and was the player able to rise to the occasion and overcome the pressure?

According to Stephen Shea in his book Basketball Analytics, "90% of teams performed worse (in terms of shooting percentages) in the clutch than in non-clutch situations." Can this me modeled? How significant is the effect?

I will try to explore this path further, looking into statistical models that would offer some elements of response to these questions.

But looking at all hat has been said it seems the debate originates from the fact that "being clutch" is never well-defined. Suppose we could at any point during a game give a score from 0 to 100 as to how good a player is. Suppose player A is at 90 throughout non-clutch times, but drops to 80 in clutch situations. Whereas player B is at 60 in non-cutch situations, but steps his game up to 70 when the game is on the line. Which is clutchier? The one with highest absolute value, or the one stepping up his game and taking the pressure head-on. Answering this would already be a giant step in the right direction.

In the meantime, please enjoy this youtube compilation of clutch shots: