Phil Simon's Blog, page 55

Event Stream Data: What is it and why should I care?

“Events, dear boy, events.”

—British Prime Minister Harold Macmillan, when asked by a reporter what would influence his government’s decisions the most.

For a long time, most business folks have thought of data in a very structured, rigid manner. This is especially true in large, mature organizations. Data was almost always stored in neat and orderly tables in relational databases. For reporting purposes, often data was sent via an ETL batch job to a data warehouse or datamart. In short, things tended to be very controlled and predictable.

To be sure, a great deal of key enterprise information still falls into this model. As I describe in Too Big to Ignore: The Business Case for Big Data the advents of Big Data and its components (the Internet of Things [IoT], sensor-driven data, social media, etc.) have not obviated the importance of clean accurate information in ERP and CRM applications. Make no mistake, though: these days events drive a great deal of critical information.

Data in Motion

Employees today can analyze massive amounts of real-time information and make better business decisions.

Perhaps it is best to define event-stream processing (ESP) against “normal” data processing. In the case of the latter, an organization would never run a “final” payroll or produce a P&L in the middle of Black Friday. What’s more, based on my experience in retail, I know first-hand that stores prepare for months for their busiest time of the year—hence the term holiday hire.

Contrast that with the suddenness with which things trend on Twitter. Although sites such as Groupon and Living Social are no longer Silicon Valley’s darlings, flash sales and daily deals can still quickly go viral. To understand and respond effectively and quickly to these events, organizations need to do more than capture the data, load it into a reporting data warehouse, and run SQL statements.

Application program interfaces (APIs) can do many amazing things in conjunction with ESP. For instance, Twitter allows developers to create exciting applications via . And the company is hardly alone in this regard. Many Wall Street firms rely upon ESP to conduct high frequency trading (HFT). To provide location-based services, telecomm providers also use ESP. (Of course, none of this happens without powerful data centers and networks, but that’s fodder for another post.)

Storage Considerations

Needless to say, relational databases simply weren’t conceived to handle information with this variety, volume, and velocity. To this end, I’m aware of very few organizations that attempt to handle ESP via traditional ETL. Most rely upon some type of cloud computing, whether it’s a public cloud, private cloud, or hybrid cloud.

Aside from the ability of newer technologies to handle ESP, cost is a major factor. Fortunately, Kryder’s Law is alive and well. Today, data storage has never been less expensive—and that will be true for the foreseeable future. Beyond cost, there’s the convenience factor. Imagine being on the floor of a retail organization and having to run to your desktop to see what’s going on. That’s so 1998. Thanks to mobile devices, employees can analyze massive amounts of real-time information and make better business decisions.

This post was brought to you by IBM Global Technology Services. For more content like this, visit Point B and Beyond.

The post Event Stream Data: What is it and why should I care? appeared first on Phil Simon.

What I Learned Writing for Inc.

What’s it like writing for one of the most popular websites in the world?

I should know. For about a year starting in March of 2012, I wrote a series of articles for Inc. (Read them here on the Inc. site if you like.) In total, I penned 23 technology- and business-related articles for the site. In this post, I’ll distill some of the lessons I gleaned from this very interesting experience.

Writing for sites such as Inc. enhances your profile, brand, and SEO. Inc. isn’t like Medium or LinkedIn. On those sites, anyone can publish just about anything. Put differently, there’s exclusivity to writing for Inc., Forbes, Fast Company, Business Insider, and the like. And make no mistake: that exclusivity enhances the writer’s credibility, something these media outlets know full well.
Writing rates are low. This stems from the first bullet point. Inc. is well aware that it offers its writers enormous exposure. As such, why bother paying them their normal rates? (Writers find out at the end of the month how many page views their posts generated and, by extension, how much money they have made.)
Along these lines, page views matter far more than the quality of the writing. This isn’t to say that all writing on these sites sucks. There’s plenty of decent and informative content. Still, Inc. and their ilk aren’t looking for the next Ernest Hemingway. Brass tacks: On sites such as these, well-written copy that generates low “engagement” will always lose to pedestrian copy with oodles of page views. I don’t like it but I understand it. That’s just the reality of the media business today.
You are not Terry Gilliam. Writers have to pitch their articles first. What’s more, articles will be edited and writers don’t have final cut.
Originality is optional. With so many business-related posts on Inc., there’s a great deal of overlap. Many new posts are essentially retreads of old ones.
People will sometimes visit your site after reading your posts. Even more than three years after my last post on Inc. ran, people continue to click through to my site. (Whether they ultimately buy my products or services, however, is another matter.) To this end, the well-trodden exposure argument isn’t totally invalid.
Your articles will be shared early and often. It wasn’t uncommon for my posts to be retweeted more than 200 times in the first few hours after going live.
Your post titles will probably be link bait—and there’s nothing you can do about it. I remember my first post on Inc.: Want to Build a Business to Last? Here’s the Secret. It’s about my fourth book, The Age of the Platform: How Amazon, Apple, Facebook, and Google Have Redefined Business. In the book and in that post, I don’t purport to know any such “secret” because none exists. You certainly wouldn’t get that impression from reading the title of that post.
Content is mostly snackable and number-based titles are common. Inc. is the antithesis of Priceonomics. Quantity is imperative at content farms, and this heavily drives the types of posts on their sites. A look at the site this morning reveals no shortage of numbers:

Click to embiggen.

There’s no shortage of people who will gladly take your place. For free. After a year or so, it became apparent that other writers fit the Inc. writing mold better than I. As a result, we parted ways. I have no doubt that the site didn’t miss a beat. As they say in football when a player gets injured, “Next man up.”

Feedback

What say you?

The post What I Learned Writing for Inc. appeared first on Phil Simon.

When WordPress’ Default Search No Longer Cuts It

Over the last few months, I have tried to land a visiting-professor position. (I’ll have more to say about that in the coming weeks.) In the process, I have stumbled across many downright appalling websites from prominent academic institutions. In some cases, I felt like I was going back in time. I’m talking about text-heavy sites rife with 10-point fonts, laughable search functionality, confusing navigation, massive margins, two-sidebar layouts, and no hint of responsiveness.

Calm blue oceans…

I’d wager that many universities and departments haven’t materially changed their websites in at least a decade. Ditto for many mature organizations. Suffice it to say that poor website design and usability don’t reflect well on them.

When it comes to website design, I subscribe to a much different school of thought. For a long time now, I have been a strong advocate of regular tweaking. (A stroll down memory lane demonstrates how my own site has changed over the years.) A website might look professional and contemporary now, but will that be the case in two years? Craigslist and Seth Godin’s blog are the exceptions that proves the rule: Websites need to evolve. They need to adapt.

x
Search Is Still Crucial

For a long time, I’ve been less than pleased with my own site in one critical way. Its default presentation of search results via WordPress and Elegant Themes left more than a bit to be desired, and even Elegant Themes admits as much.

Click on the images above to embiggen them.

What specifically was wrong with the previous search results? They’re accurate (at least I think), but there’s a startling dearth of context and metadata. It begged natural questions such as:

Is this is a post or a page?
If the former, then to which categories did the post belong?
When was this post written?
In a nutshell, what is it about?

In short, it was very un-Googley.

Sure, anyone could click on a result and answer these questions, but that process only increased the amount of friction. Put bluntly, friction is just not acceptable on websites. They should make it as easy as possible for users to find what they want. Quickly. Period. (For more on this, see Don’t Make Me Think, Revisited: A Common Sense Approach to Web Usability [affiliate link].)

Admittedly, any site with a decent amount of content faces challenges in this regard. (This site currently sports roughly 1,100 posts.) Yeah, tags, categories, related posts, and taxonomies all certainly help. Still, it’s folly to think that search today is superfluous. Google is still printing money. Why any site wouldn’t prominently display a search bar or icon is beyond me.

Why any site wouldn’t prominently display a search bar or icon is beyond me.

To this end, I’m pleased to announce much-enhanced search functionality on this site. Aside from providing more context and metadata for each query, the styling far exceeds its predecessor. Major props to Monterey Premier (affiliate link) for taking search to the next level.

Simon Says: Search still matters. Big time.

I won’t proclaim to be an expert on usability and user experience, but I know a thing or two about good website design. Amazing search functionality by itself will not guarantee page views, stickiness, piles of cash, and the like. Content is still critical. Still, poor or nonexistent search functionality has never helped any site gain traction.

Feedback

How good is your site’s search capability?

The post When WordPress’ Default Search No Longer Cuts It appeared first on Phil Simon.

How to Handle Support for Decommissioned Enterprise Apps

Microsoft recently nixed support for SQL Server 2005. You might not care about the announcement if your organization runs dB2, Oracle, MySQL, or another backend database. Also, if it had upgraded from the 2005 version over the past decade, the news probably didn’t matter much to you.

Still, odds are that many organizations continue to run legacy versions of the database. Case in point: Even though Microsoft has been touting Windows 10 for a while now, many companies never migrated their employees from Windows XP. This is astonishing, since XP launched on October 25, 2001. As an interesting aside, nearly all automated teller machines (ATMs) continue to run XP.

Work in enterprise IT long enough, as I have, and it’s only a matter of when—not if—your organization faces a similar dilemma. At some point, a software vendor is going to stop providing support for key enterprise applications and technologies. In this post, I’ll discuss the main options for organizations to consider.

Options

The six main options for organizations facing this dilemma include:

Go naked. In a nutshell, this entails doing nothing. It’s a high-risk stratagem that I have never recommended to my clients. Very rarely does this ever make sense.
Purchasing and deploying an entirely new technology. This is no small task. As I wrote in Why New Systems Fail , moving from a database or CRM or ERP suite is fraught with peril.
Procuring independent third-party support. Many organizations have taken advantage of this option, particularly for large software vendors. For instance, Rimini Street represents a viable option for Oracle clients.
Upgrading to a new(er) version for the applications, databases, and programs. This is often the wisest move. Few if any software vendors suddenly refuse to support a major software release. They generally communicate at least year ahead of time of their plans and, as I’ve seen, will move dates if clients object strenuously enough.
Asking the vendor for an exception. This Band-Aid approach often involves cutting a check for “sunset” support. There’s still no such thing as a free lunch. Also note that CIOs can only go to the well so often. That is, don’t expect software vendors to routinely make exceptions. It simply doesn’t make sense for them allocate staff who primarily maintain older versions of products. This is doubly true in the case of SQL Server 2005.
Hiring in-house developers and support folks to handle bugs, patches, and fixes. I’ve seen a few organizations do this in my career. Generally speaking, it’s not for the faint of heart. It’s a resource-intensive endeavor. Still, depending on the extent to which the organization wants to customize its software and maintain legacy applications, it might make sense,

Think about your options well before the expiration date.

It’s important to note that the support ramifications of each option hinge upon the type of infrastructure involved. That is, an organization may face different costs and resource requirements depending on where an application ultimately resides (re: on-premise vs. through a third-party’s cloud-computing service).

Simon Says: Think about your options well before the expiration date.

There’s no easy answer to this conundrum, but a responsible organization cannot let key enterprise apps go unsupported. Think about the options in this post well before a software vendor pulls the plug.

Feedback

What say you?

This post was brought to you by IBM Global Technology Services. For more content like this, visit Point B and Beyond.

The post How to Handle Support for Decommissioned Enterprise Apps appeared first on Phil Simon.

Dataviz 101: Who’s Your Audience?

By now, most intelligent business professionals accept the fact that Big Data and dataviz have arrived. To quell any remaining doubt, consider the recent words of a Gartner analyst: “Hype is now being replaced by practicality.”

Of course, that doesn’t mean that everyone in an organization can and even should start setting up Hadoop clusters and learning Python. As I write in The Visual Organization, data visualization has become essential to not only understand what is happening, but to act on it.

Development Details

Developing an intelligent dataviz certainly requires some forethought. One should not sit down to create a data visualization in isolation. Think about it. An architect would never draw elaborate blueprints and start building without consulting their clients. The lifestyle and needs of a relatively young family of four often contradict those of seventy-something retirees. By the same token, data-visualization experts and practitioners know that when building tools one size rarely fits all. Complicating matters further, there’s never been a greater variety of tools available for developers.

This begs the question, how should developers design data visualizations that will be useful for different audiences? In this post, I’ll describe the major types of business audiences that typically interact with developers. I’ll also provide some tips on how to effectively design and develop dataviz tools for important groups of business users.

Interactivity should be the rule, not the exception.

Executives and business users. As I know all too well, these non-technical folks are the antithesis of conventional “techies.” Many even are dataphobes. To this end, it’s imperative to ask them high-level questions and ensure that they are being clear with their requirements. Often, because of their backgrounds, the greatest challenge for developers lay in communication and ensuring that business users and developers use a common technical and dataviz vocabulary. Using one set of terminology can help ensure that visualizations address most key data-related questions.


Product and project managers. Unlike the top brass and the 30,000-foot crowd, these folks often try to solve very specific problems and address specific issues. What does this mean for dataviz? It behooves developers to work with these groups to select a charting library that will meet their short- and long-term visualization needs. Is the selected tool flexible enough to work with current requirements? What about anticipating future ones?
Data scientists. Developers would do well here to err on the side of flexibility and size (re: Big Data). A great deal of data science hinges upon exploration and data discovery. That is, true data scientists will follow the data wherever it goes. At the same time, though, they don’t know where the data will take them. As a result, developers who create a tool that lets data scientists get hands-on
Software developers. Generally speaking, developers are quite the technical bunch. They can “go deep” with different concepts and despise rework—especially when it can be avoided. Build with the understanding that further integration with different application programming interfaces (APIs) is not only possible. It’s expected and easily accomplished compared to prior years.

To be sure, these audiences differ on many levels: their needs, backgrounds, levels of technical proficiency, and the like. Still, it’s not hard to find a commonality or two among them; in other words, it’s wise to build a tool that can provide a number of different views of the data. For starters, it’s best for dataviz developers to build for interactivity:

Simon Says: Interactivity with data visualizations should be the rule, not the exception.

These days it’s not very hard for skilled developers to make even simple, stacked column charts interactive and embeddable. This way, their audiences can use a single dataviz to ask and answer a number of questions. Put differently, as intelligent folks and firms have started to realize, interactivity with data visualizations should be the rule today, not the exception.

Feedback

What say you?

This post is brought to you by ZingChart’s JavaScript charting library. The opinions in this post are my own. The ZingChart folks love charts. Feel free to hit them up for your dataviz project.

The post Dataviz 101: Who’s Your Audience? appeared first on Phil Simon.

Disrupted: My Misadventure in the Start-Up Bubble

What do you get when you put a fifty-something ex-Newsweek reporter in a startup environment rife with new, mostly white college graduates? I was curious, so I read Dan Lyons’ How about Disrupted: My Misadventure in the Start-Up Bubble.

Based on the title, subtitle, and excerpt I had read, I knew how the book ended. Still, right off the bat Lyons hooked me with his outrageous tales from his time at marketing unicorn Hubspot. Faced with few opportunities in traditional media and two young children, Lyons takes a position as a nebulous “marketing fellow.” Almost immediately, he finds himself completely out of place at a company plagued by horrible jargon (delightion and 1+1=3 made me die a little inside). And there’s plenty of toxic internal politics in this overly zealous corporate culture—with an emphasis on cult.

Honest and Scathing

I can see how Lyons’ style can rub some people the wrong way. He’s not above throwing former colleagues under the bus and his language is often colorful. As someone who has worked in similar milieus, though, his observations certainly reminded me of some professional days that I’d rather forget. Lyons spares no one, but Hubspot’s (mostly female) blogging team comes across as particularly puerile and vindictive. Ditto for “Trotsky”, his former boss.

I highly recommend the book to people curious about what could very well happen to them.

Lyons’ tone throughout the book isn’t necessarily bitter, but he doubtless holds a few grudges about the way that his former colleagues and bosses treated him. I would too. For the most part, I found him humble and self-reflective. At times, though, he is downright scathing. Against this backdrop, a few Hubspot execs resigned over a potential hacking scandal related to this book, something that certainly will help Lyons’ book sales.

It would be incomplete to classify Disrupted as merely an Office Space-esque critique of Corporate America. It also serves as important social commentary about the way that more senior employees are viewed and valued in a hyper-agressive startup culture hell bent on an IPO. In other words, you will both laugh and think.

I consumed the book in less than a day and highly recommend it to people curious about what could very well happen to them.

Disclaimer: Hachette Books sent me a free copy for a potential review or interview.

Originally published on HuffingtonPost. Click here to read it there.

The post Disrupted: My Misadventure in the Start-Up Bubble appeared first on Phil Simon.

Throwing Rocks at the Google Bus

Amazon has rarely been in the black over its twenty-year history, yet its investors have generally been ecstatic. Why? The company’s sales continue to rise and so has its stock price. One could argue that Wall Street wouldn’t be hammering Twitter so hard if the social network could break its nagging user-growth drought. For their part, venture capitalists usually emphasize growth over profits for early- and even middle-stage startups.

In an of itself, growth isn’t a bad thing, but perhaps we value it too much and at the exclusion of other even more important things. The consequences of this “growth at all costs” mind-set are severe. At least that’s the contention of Douglas Rushkoff in his new book Throwing Rocks at the Google Bus: How Growth Became the Enemy of Prosperity (affiliate link). 

Rushkoff presents an interesting and provocative critique of the current state of affairs in his expansive text. The title stems from highly publicized protests against Google’s private buses in San Francisco and related social unrest. (For more on this, see the disappointing HBO documentary San Francisco 2.0.)

Throwing Rocks at the Google Bus contains plenty of excellent research, examples, and general pontificating. To be sure, the book is certainly informative. Rushkoff devotes significant passages to the evolution of currency, financial services/employee retirement, and the growth of the modern-day corporation. In so doing, he forces readers to ask disconcerting questions such as:

Is today’s tech-infused capitalism running out areas to grow?
Is creative destruction really destructive destruction?
Are corporations more important than individuals and society?
What are the downsides of the “extractive qualities of industrialism”?

x
Platform Monopolies

I didn’t concur with every one of his points and I took his perspective with more than a grain of salt

Rushkoff can be a bit verbose and his language can be clunky. What’s more, at times he overstates things, particularly with regard to what he calls “platform monopolies.” For instance, he writes about how Amazon is currently the leader in cloud-computing services, but will that always be the case? Regardless of whether these companies qualify as monopolies, I’d argue that they face much more competition than he contends. (With respect to cloud computing, Microsoft, Google, and Rackspace are major players.) More generally, history contradicts the notion that today’s powerful and successful companies will never falter. Disruption takes place faster than ever. Just ask Yahoo!, MySpace, IBM, Dell, HP, Microsoft, and other behemoths that once seemed incapable of faltering. (Of course, whether even more powerful companies ultimately replace today’s incumbents is a separate discussion.)

Parting Thoughts

I didn’t concur with every one of his points and I took his perspective with more than a grain of salt. Still, it’s essential to read things with which you don’t necessarily and entirely agree. Without question, Throwing Rocks at the Google Bus makes you think about fundamental questions regarding the present and the future.

Disclaimer: Portfolio sent me a free copy for a potential review or interview.

Originally published on HuffingtonPost. Click here to read it there.

The post Throwing Rocks at the Google Bus appeared first on Phil Simon.

What’s the Future of Big Data?

It’s impossible to pinpoint a specific date on which this occurred, but it’s fair to say that Big Data has arrived in earnest. (My book Too Big to Ignore was published more than three years ago.) Sure, there’s still plenty of hype, but Big Data has moved from theory to practice in many organizations. As a Gartner analyst puts it, “Hype is now being replaced by practicality.”

Put differently, we know where we are now, but where are we going? What does the future hold for Big Data?

Analyst Forecasts for the Big-Data Market

If you’ll pardon the pun, it’s easy to see that Big Data has become a big deal. Forget sky-high valuations of startups and money-hungry venture capitalists. Big Data will add significantly to US and world employment. In the words of Peter Sondergaard, a Gartner senior VP and global head of research:

By 2015, 4.4 million IT jobs globally will be created to support Big Data, generating 1.9 million IT jobs in the United States. In addition, every big data-related role in the U.S. will create employment for three people outside of IT, so over the next four years a total of 6 million jobs in the U.S. will be generated by the information economy.

Plenty of other reputable organizations confirm the importance of data-related skills. For instance, LinkedIn’s hottest skills reiterate what many recruiters already know. For the foreseeable future, labor markets will richly reward those with the ability to make sense of rapid and increasingly complex streams of information.

Vendor Landscape

That’s not to say, though, that everyone will profit, much less equally. To label the current vendor landscape crowded is the acme of understatement. Just take a look at the following graphic:

Click image to enlarge it.

The current state of affairs is beyond busy; it’s downright chaotic. The word consolidation comes to mind. Legacy technology companies have already started to gobble up promising startups. To be sure, niche players will ultimately remain, but it’s folly to think that every company that purports to “do” Big Data will wind up as the next HortonWorks or Cloudera.

What should companies be doing now to get ready and not fall behind?

Before continuing, I certainly don’t own a crystal ball. What’s more, I’m not privy to all companies’ current and future plans. (If I were, I wouldn’t be able to talk about them anyway.) Still, a few cautious recommendations are in order:

Realize that there’s no guarantee. At some point, all companies will have to make educated bets on strategies, future directions, technologies, and partnerships.
There will be more than one seat at the table when the music stops playing. Don’t expect “one winner” to emerge. After all, several different relational-database applications have dominated the market for some time now. Nothing about Big Data indicates that one size will fit all.
Pay attention to the Internet of Things (IoT). While still in its infancy, the IoT is coming. Right now, security and privacy concerns and the lack of standards hamper its adoption. Make no mistake, though: At some point in the near future, myriad devices will be able to seamlessly communicate with each other. Serious players in the Big-Data world are preparing for that reality.

Feedback

What say you?

This post was brought to you by Analytics@American, the online Master of Science in Analytics from The Kogod School of Business at American University, and the opinions are my own. The program offers concentrations in Business and Policy, Financial Analytics, and Marketing Analytics. It requires no standardized test scores or minimum professional work experience to apply. To learn more, click here.

The post What’s the Future of Big Data? appeared first on Phil Simon.

Why Big Data Means Big Salaries

For a while now, I have been dispensing career advice on this blog and as a coach. Sometimes, my recommendations come in unexpected situations. For instance, after keynoting in San Diego last year, I had the pleasure of signing copies of The Visual Organization for more than 200 conference attendees. One of them told me that he worked as an adjunct professor. He then asked me the following query as I signed a copy of my book:

How do I get my students interested in statistics and analytics?

By way of background, post-keynote book signings tend to be rapid-fire events. Everything is a blur, and I said the first thing that popped into my otherwise-engaged mind: Show them data on starting salaries for people with those skills.

I could see the light bulb illuminate over his head. He smiled, thanked me, and walked away very pleased.

Supply and Demand Are Out of Whack

I won’t take credit for any great insight here. It doesn’t take a rocket surgeon, soothsayer, or labor economist to understand the growing importance of data science, analytics, statistical analysis, data visualization, and their ilk. Big Data is crossing the chasm. Industries now using it include healthcare, oil and gas, banking and securities, telematics, and many more.

Want more proof on the arrival of Big Data? A simple of search of LinkedIn’s hottest skills confirms what many recruiters already know, but why is this happening now.

Why Now?

As with any hot labor market, there’s a fundamental imbalance between supply and demand. Wages are on the rise because there are too few people for too many jobs. For instance, data scientist is one of the hottest job titles in the country right now. In a recent report, McKinsey estimated that the U.S. will soon face a shortage of approximately 175,000 of them. Generally speaking, people who know how to make sense out of Big Data will be able to make nice livings for the foreseeable future. Tools include Hadoop, HBase, Hive, and many more. (For more on this, see Too Big to Ignore.)

This trend may seem sudden and begs a natural question: Haven’t organizations always demanded employees with strong quantitative skills?

Certainly not across the board and not nearly to the extent demanded today. To state the obvious, intuition and gut feel dominated many if not most business decisions during the Mad Men era.

x

To quote Melvin Kranzberg, “Technology is neither good nor bad; nor is it neutral.”

That’s no longer the case. In many organizations, numeracy is becoming a sine qua non. The geeks have inherited the earth and, as Mark Andreessen famously said, software is eating the world. Case in point: Take a quick look at the eye-popping valuations of AirBNB, Uber, and Lyft reveals that most traditional industries are being disrupted.

This was a white-hot topic at the recent World Economic Forum in Davos. I’ve said this many times: Every company is becoming a tech company. Some just haven’t realized it yet. Perhaps the Fourth Industrial Revolution is here after all.

Simon Says

To quote Melvin Kranzberg, “Technology is neither good nor bad; nor is it neutral.”

What does this well-worn maxim mean for you? It depends. Those with coveted data and technology skills will flourish in this new era. Hidebound organizations and individuals will find themselves holding the short straw.

Feedback

What say you?

This post was brought to you by Geotab Fleet Management Solutions. The opinions in this post are my own. To learn more about best practices for improving business productivity, safety, and compliance, visit Geotab’s blog.

The post Why Big Data Means Big Salaries appeared first on Phil Simon.

What are the odds that the Warriors go 4 of 30 again?

Without question, the Golden State Warriors are the talk of the NBA this year. The team is on pace to eclipse the Chicago Bulls’ 72-10 record in 1995-96. Yet, the lowly Lakers beat the Warriors on March 6 in one of the greatest regular-season upsets in NBA history.

Watching the game, I couldn’t believe how poorly the Warriors shot from behind the arc. (The team made only four of 30 attempts.) The Warriors’ shot chart in on the left, courtesy of ESPN:

As you can see from the chart below, this was by far Golden State’s worst TPA performance this year:

Click to embiggen.

The Warriors’ three-point shooting on March 6 was the very definition of the term anomaly.

In case you’re wondering, it’s not hard to find this information. I grabbed individual game data from basketball-reference.com and then simplified the dataset a bit. (Download it here.)

As you can see from the chart above, the team’s three-point percentage against the Lakers on March 6 was by far its worst of the year. But, given its three-point proficiency this year, what were the odds that the Warriors would make so few shots on so many attempts?

Disclaimers

A few disclaimers are in order here. First, for the sake of simplicity, I’ll assume that all TPAs are independent probabilities like coin flips. That is, I won’t get into the hot-hand fallacy here. Second, I’ll ignore any defensive strategies or adjustments by Lakers’ coach Byron Scott. It’s not hard to devise a scheme that effectively makes it difficult for the Warriors to shoot effective TPAs. In the extreme, Scott could have had all five Lakers guard the three-point line and willfully conceded layups and other easy two-point shots.

To answer my question, I used a statistics tool that I learned in college: the binomial distribution. It’s perfect for roulette and coin flips. For instance, to determine the odds of three heads in ten coin flips:

What are the odds that the Warriors go 4 of 30 again?

Fortunately, with the data, it’s easy to figure these things out. I used an Excel function.

Odds that Warriors make exactly 4 three-pointers on 30 attempts: 0.07%
Odds that Warriors make at least 4 three pointers on 30 attempts: 99.67%

In other words, even with the limitations stated above, the Warriors’ three-point shooting on March 6 was the very definition of the term anomaly. It’s extremely unlikely that we’ll see anything close to those numbers from Steph Curry and company this year. In fact, in the Warriors’ next game on March 7 against Orlando, the team hit 16 of 35 TPAs (45.7%), a number much more in keeping with its performance this year.

In case you’re wondering, by this rationale, the odds that the Warriors miss 30 consecutive threes are one in roughly 10.7 million. With a 42 percent success rate:

(0.58)30

The post What are the odds that the Warriors go 4 of 30 again? appeared first on Phil Simon.