8 Quotes That Shaped My Worldview

silhouette photography of person

A worldview is a collection of attitudes, values, stories, and expectations. It represents our most fundamental beliefs and assumptions about the world around us. “As such, worldviews play a central and defining role in our lives. They shape what we believe and what we’re willing to believe, how we interpret our experiences, how we behave in response to those experiences, and how we relate to others”, says James Anderson of Ligonier Ministries.

These thoughts have greatly shaped how I see the world.

“The fool doth think he is wise, but the wise man knows himself to be a fool.” (Shakespeare)

It’s easy to get caught in the trap of thinking we know what is best for ourselves, for others, and the world. The best we can do is have some humility the next time we catch ourselves thinking we know the way. It’s okay not to have all the answers – better than okay. Acknowledging this may be our only chance at wisdom.

“Security is mostly a superstition. It does not exist in nature, nor do the children of men as a whole experience it. Avoiding danger is no safer in the long run than outright exposure.” (Hellen Keller)

There is no “safe”. Everything you have could be gone in a minute for reasons beyond your control, whether you play it safe or not. If you knew that savings account you’ve been contributing to for so many years would be wiped out when you needed it, would you regret not having taken some risks along the way? That is not to say to be reckless, but being risk averse doesn’t pay the life long dividends you think it does, and likely steals the best version of yourself away.

“You shouldn’t hesitate to modify your perceptions to whatever makes you happy, because you’re probably wrong about the underlying nature of reality anyway.” (Scott Adams)

Part of why we’re so hard on ourselves is we think we understand the world. We think we know its rules and limitations, and our own potential or lack thereof. But what if we’re wrong about all of that? What if we’re just not capable of understanding the true nature of our world or how others perceive us?

Scott continues, “If I had to bet my life, I’d say humans are more like my dog trying to use psychic powers on me to play fetch than we are like enlightened creatures that understand their environment at a deep level.” This worldview helps me not to take things so seriously. I believe there is more magic, wonder, and possibilities both within the universe and within ourselves than we are capable of seeing or understanding.

“Believe those who seek the truth, doubt those who find it.” (Andre Gide)

Confirmation bias is an inescapable part of humanity. We don’t seek the truth so much as we seek out things that support the conclusions we’ve already made. It’s sad that it takes so little to form an initial opinion and that all information seen after developing this opinion is seen as either “right” and in support of our early conclusion, or “wrong”. This is true even in the field of data science where we can masterfully manipulate the data into telling whatever story we want. And just being aware of confirmation bias doesn’t make us immune to it. I hope to be as open-minded to concede to the possibility that everything I know might be wrong, but I’m probably too entrenched in my views to truly be that objective.

“I find it helpful to remind myself that every human is a mess on the inside. It’s easy to assume the good-looking and well-spoken person in front of you has it all together and is therefore your superior. The reality is that everyone is a basket case on the inside. Some people just hide it better. Find me a normal person and I’ll show you someone you don’t know that well. It helps to remind yourself that your own flaws aren’t that bad compared with everyone else’s.” (Scott Adams)

Enough said.

“Be, Do, Have.” (Robert Kiyosaki)

Before you can DO what is necessary to HAVE the life you want, you must BECOME the person who would do those things. The order is very important. I used to think if you do certain things you can have what you want and that will make you into the person you want to be. But having things doesn’t make you into anything, nor would you likely do enough to have what you want without first becoming the person you need to be.

The “be” is a flywheel. If you’re a _____ kind of person, you can’t help but do _____, and if you do that you can’t help but have _____. It’s automatic. You can’t help but do what you do because of who you are. This can work to your benefit or against it, depending on what your “be” truly is.

“Life will give you whatever experience is most helpful for the evolution of your consciousness. How do you know this is the experience you need? Because this is the experience you are having at the moment.” (Eckhart Tolle)

Have a difficult person in your life? I believe this is a gift, handcrafted and placed exactly into your life so that you can learn from it. Sometimes it can take years to understand the lessons from the difficult things in your life, maybe a lifetime, and life will keep serving you the same kind of lesson until you get from it what you need.

“For even a few moments at a time, being aware of what is arising, gives us the possibility to make a choice.” (Joseph Goldstein)

I’m not so sure we actually have free will, or maybe that we are not really conscious the vast majority of the time. So rarely are we present enough to make a choice we could truly call our own. The best we can do is focus on the present moment, notice our own thoughts as they arise and maybe instead of just reacting and following our program, choose if those desires are really from our innermost soul.

See also Why You Probably Don’t Have Free Will.

Things That Trigger Me (Well, Tech Related)

After I left my last company, one of my teammates shared with me a list of things she had been keeping about me in the Notes app in her phone called “Things that trigger Doug”. It must say something about me that someone would keep such a list. Anyway it made me laugh! You’ll only appreciate this if you work in tech. I give it to you unedited:

  • Writing shitty code the first time with the excuse of “trying to get it to work” / hard coding shit to make it work
  • White IDE theme
  • Stealing his phone
  • Waiting in a line for 2 hours
  • When you don’t believe in containers
  • Windows 7 in 2019
  • Using private chats instead of public Slack channels
  • Agile words
  • People who wait to write their Dockerfile until the last minute /don’t dev in a docker container
  • People cheating in the step challenge when he runs 14–16 miles a day
  • Designated CAB meetings /calling releases “emergency” or “off cycle”
  • Notepad++
  • QA tester boxes / manual testing
  • Ad blocker. Support the creators!
  • Naming things poorly
  • End to end tests
  • People calling stuff a blocker when they aren’t blocked by shit
  • Iframes
  • Wasting money in the name of frugality
  • Putty
  • Putting “Personal Interests” on your resume

10 Rules For Building Better Internet Applications

blog_headerIf you’re working on an application that’s been around for a while, has lots of usage and several different teams working on it, there’s a good chance it’s a dumpster fire. It happens to the best of us. Building software with real customer adoption is messy business. While there is no one size fits all, these are my opinions on building more reliable, scalable, and maintainable Internet applications.

1. Go All In On One Cloud

Pick the leading cloud provider that makes the most sense for your team and go all in. Don’t miss half the benefit of the cloud by limiting yourself to the lowest common denominator between cloud providers, which is basically just VMs. This is folly anyway as it is nearly impossible to build an application natively in the cloud without tentacles into proprietary resources from that provider. Trying to be on the light end of the lock in spectrum is not worth the trade off you’d be making in lost productivity and reliability.

2. Forget About Servers

Use containers and/or serverless for 100% of your dev, stage, and prod environments. There is almost no reason to use VMs directly for any new development. While containers and serverless have their own set of challenges they are an order of magnitude better than using configuration management (Ansible, Chef, Puppet) and are your only hope for getting your dev team to participate in DevOps, which is necessary to scale development of a cloud native application.

3. Commit Wholly To Infrastructure As Code

Don’t take shortcuts that question your ability to stand up the application from source in the future. Make defining your infrastructure as code a requirement, not a nice to have. This applies to your infrastructure primitives (e.g. building blocks directly in your AWS account) as well as any third party tools or services you may want to integrate with. If it can’t be brought to life entirely from source code, you should seriously consider an alternative that can. Your application code needs a specifically configured environment to run in. Setting up that environment by hand is like writing a program, deploying it to production, then deleting the source code. Let’s not even talk about that install guide document you pretend is accurate and up-to-date.

Your first line of code should be your Dockerfile, or maybe your serverless yaml or your Terraform. The point is, start with your infrastructure code, at least enough to define your immediate development environment, which becomes the blueprint for production. Store your infrastructure code with your app code, lock step with the application in the same git repo with each microservice. You should be able to clone a git repo from any computer, build the Docker image and immediately start working on it, no matter what language it’s written in or dependencies it has. “It works on my machine” is the joke of the last generation of software development. Think of your machine as a text editor and a Docker engine.

4. Embrace The Ephemeral

I think it’s helpful to think of your code running on a “server” that didn’t exist 5 minutes ago and won’t exist 5 minutes from now (or maybe seconds, or milliseconds). If you haven’t noticed, in the cloud they aren’t even called servers anymore, they’re instances, hinting at their proper use lifespan.

Never write out any state to local disk volumes. Pretend your environment doesn’t even have a local disk. Expect instances of your program to terminate abruptly due to auto-scaling or chaos experiments. Your operations need to be idempotent, and anticipate retries. You need good logging and usage metrics, which your application must be deliberately broadcasting. Throw away your ssh keys and keep the telemetry mindset of your application running on an inaccessible remote location.

This forces the issue of immutable infrastructure. It makes no sense to modify or “patch” something that has a lifespan of milliseconds to minutes. Change is always by terminating the existing resource and replacing it with a new one with new settings.

5. Use Managed Services Wherever Possible

Building and operating clusters of anything isn’t providing business value or helping you to quickly iterate. Wherever possible, skip all that undifferentiated heavy lifting and use a managed service.

Installing a database server for example, on a single server for development isn’t so hard, but doing it in production in a way that is highly available is another story entirely. You have to setup an auto-scaling configuration where nodes can launch and rejoin the cluster, setup and test failover and restore for your HA configuration, and setup and test all the associated monitoring and alarms. It’s easy to get something wrong here and since you’re usually in the data layer, these are costly mistakes. And since our dev and stage environments need to match our production environment exactly (apart from instance size), the easy single server installation isn’t acceptable in any environment. Maybe this would be manageable if we were doing this once for one huge database server, but in a microservices world each with their own isolated data layer, it’s just not practical to self manage with dozens of services. Recalling the earlier rule about bringing everything to life entirely from infrastructure code, usually the only way to accomplish this without a herculean effort is to use a managed service. If you’re insisting on installing everything on EC2 yourself you are missing half the benefit of the cloud.

6. Testing Should Make You Go Faster Not Slower

Testing should give you rails so you can speed, not gates to force you to come to a stop. Your tests (all of them – unit, integration, load, front end) should be a completely automated part of your build and deploy pipeline, and like your infrastructure code, should reside in the same git repo as the component they are testing. The same integration tests should run against all environments, especially production, which is the most important environment to test because it is the only one your customers actually see. This means you need to use test accounts in your real databases, your testing can’t rely on starting from a fresh database each run, which isn’t feasible in production, and demands exclusive use of an environment for testing, which obviously doesn’t scale. This requires a close partnership with development, it simply won’t happen if you treat QA as a siloed activity.

Testing should share the microservices mindset – when we replace a pipe in the basement, we don’t always need to reinspect the whole house including the windows in the attic. Testing is critical, but we don’t want testing at any cost. Don’t let outdated testing practices be the weak link in your agile process.

7. Always Be Deploying

Because I think this one is so important, I’m going to borrow the sentiment from the infamous Glengarry Glen Ross, “Because only one thing counts in this life. Get [the code deployed to production]! You hear me you ***** *****! A, B, [D]. A – always, B – be, [D] – [deploying]. Always be [deploying]. Always be [deploying]!”

The more time that passes between deployments, the higher your integration risk, and the greater chance you are blocking another service which you are a dependency for. This is exactly why we do CI/CD in development. While I’m not saying you need to do continuous deployment in production, the same risks accumulate as time goes on so you want to keep the change delta as low as possible. Plan on deploying every sprint. Most deployments are back-end services with minimal to no customer facing aspects, so this doesn’t necessarily mean visible UI changes all the time.

Each microservice should be independently testable and deployable. When working on a feature that spans multiple microservices, deploy each one as soon as it is ready, don’t squirrel them away to deploy all at once in some blaze of glory.

So put that coffee down, coffee is for deployers!

8. Write Your README Before Writing Any Code

As the saying goes, “If you don’t know where you’re going, any road will take you there.” I love the Amazonian practice of starting a new project or feature by first writing a press release announcing it. I think that causes you to think bigger and to go into the work with a more clear picture of the most important parts of what you’re building. Perhaps the microservice equivalent of that is writing your README first. Give a couple sentences about what the thing does. Show usage examples demonstrating the main interfaces. Show examples of how to build, test, and deploy it. Do this first, in the README in your git repo, not last in some external document management system.

And for the love of God, before any code is written, think about how you’re naming things. Nothing shows lack of thought about how your service fits into the bigger picture of your ecosystem of services like giving it a vague or misleading name. Renaming things seldom happens, or is really expensive to do later.

9. Open Source From The Beginning

Even if you don’t want third party contributions, I think open sourcing your project reinforces a few helpful perspectives. First you are forced to handle secrets appropriately, which makes it a lot easier to get a short term consultant involved, because you can actually share your code base without worrying about changing the secrets later (let’s be honest, doesn’t always happen). Open sourcing your project may also inspire confidence from your customers about the longevity of your platform and inspire the best from your developers who will hold themselves to a higher standard knowing their work will be on display.

More importantly, it serves as a constant reminder that while it takes valuable skill to produce the source code, it has no intrinsic value and can be easily duplicated. I know this is a difficult pill to swallow when you’re spending boatloads of money on development but it’s the truth. Your value is in your data set, relationships, reputation, and execution.

At least look to open source components of your application which may be general use tools, or with a small amount of additional thought could have a general use interface rather than being specific to your application.

10. Iterate on Customer Experience

Andy Jassy ended his 2018 AWS re:Invent keynote with some remarks which encapsulates what this is all about.

“For any of us who are trying to build long standing sustainable businesses, which is hard, the most important thing by far is to listen really carefully to what your customers want from you and how they want the experience to improve, and then to be able to experiment and innovate at a really rapid clip to keep improving the customer experience. That is the only way that any of us will be able to survive over a long period of time. That’s what you should care about, giving your builders the most capable platform and allow them to keep iterating on that customer experience.”

While of course he’s suggesting that AWS specifically be the foundation of that platform, the sentiment rings true, and that’s what all of this is ultimately about, investing our resources iterating on customer experience, and not get bogged down with boring IT stuff.

Super Powers: AWS re:Invent 2016

Amazon knows how to throw a party. They hired the best DJ in the world, literally – Martin Garrix. The re:Invent Party was an experience I’ll never forget. Walking in, you are handed an activity map, showing the locations of the digital lounge, beer garden, dance floor, and play zone including a laser dome, ball pit, bubble soccer, climbing wall, and a labyrinth. But once I got to the DJ area, I was mesmerized.  I stood in the crowd and soaked in the music, the lights, and the special effects. It was truly a performance. Even though I traveled alone and didn’t know anyone there, I felt connected, surrounded by thousands of other like minded people celebrating at this amazing event.  AWS CEO Andy Jassy captured the sentiment best saying simply “it is awesome to be here.”

The conference was sold out at 32,000 attendees and occupied most of the Venetian and the Mirage resorts.  Everywhere you went it seemed like Las Vegas was invaded by swarms of cloud enthusiasts all wearing the black and blue AWS hoodie given to us on our first day.  Fredric Paul from New Relic put it well saying “The whole event has the palpable buzz of a celebration of a successful, growing industry that feels like it’s on the right side of history.”

The theme of Andy Jassy’s keynote was that AWS gives you super powers, and I think it’s true. You have access to essentially infinite resources. As a single developer you can build something over the weekend that you can deploy globally, that can auto scale to handle any amount of volume, store any amount of data, and initially cost next to nothing.  Want to cut your teeth on a 128 CPU server with 2TB of memory?  Spin it up right now, it’s yours for $13.34 per hour.  Where else would most of us ever even have access to a machine like that?  How about a fleet of them on the spot market for up to 90% off?  Need to accumulate 100GB of storage per day?  No problem, write it out to S3 for just $0.03 per GB per month, and know you’ll never run out of storage space.  Werner Vogels, CTO of AWS put it this way – “One of the things the cloud has done is democratize access to technology power.” Just knowing that I can affordably access any resources I need challenges me to tackle something of scale.

Better yet, as Jeff Lawson, CEO of Twilio, astutely pointed out, “you don’t get points for using servers, you only get points for serving users”.  With serverless architecture, you can put together entire solutions on inherently scalable building blocks that can serve millions of users, but only cost as much as a Starbucks coffee until it starts to get some traction. The cloud really has given us super powers.

Jassy wrapped up his keynote with a few words on the next 10 years. Because AWS has grown so substantially in its first 10 years, would the the next 10 will have less invention? “I believe the next 10 years will have markedly more innovation,” Jassy said.  Apparently, the cloud is just getting started.  Instead of debating wheather companies would use the cloud or not, Jassy said “They will spend all their time using the cloud and its massive capabilities to build new business that we never imagined before.” He continued saying “This is a world that I want to live in. And this is a world that I think is very exciting to everyone in this room.” I had a blast at re:Invent, and left pumped up and ready to go build something.

DevOps, Because We’re Not In Kansas Anymore

DevOps is one of those things that are hard to wrangle down to a definition, and if you’ve read anything on the topic you probably didn’t walk away with a concrete understanding of what it is.  Most definitions seem incomplete if not even a little misleading. In particular, the main thing you hear is that DevOps is about increased collaboration between dev and ops teams. However many apps in the cloud are supported entirely by the dev team, with minimal or no ops involvement, but still very much have need for DevOps. In this context, DevOps is more about new tools, methods and practices of building and running apps in the cloud.

“I Would Spend 55 Minutes Defining the Problem and then Five Minutes Solving It”
Albert Einstein

Okay Einstein, let’s start there. The problem is we need to run and continually update our software in a scalable, reliable, and affordable manner. For this we look to the cloud. But running applications in the cloud is fundamentally different, and to do it effectively we need to adapt to a new way of doing things.

DevOps is the “stuff” that solves that problem.

So what is DevOps? The predominant, and tip of the iceberg answer is that DevOps is about delivering software faster, which in turn is about automation — having a robust continuous integration and testing process, automating the deployment process, and even automating the creation of infrastructure. It’s getting from “I just finished writing code” to getting it into an environment to test.

Before the cloud, there was often a sense of permanence and consummation in both the infrastructure and the application itself. The new reality is that both are ephemeral, subject to constant change and updating.

We do DevOps because we’re not moving into a house, we’re getting on a treadmill.

So sure, DevOps is about building a deployment pipeline, but that’s really just the beginning.  It’s the “day two” stuff, the operations, the scaling and monitoring, that I think is often overlooked as part of DevOps. To consider why we need new tools and methods, let’s reflect on what many of us were doing prior to the cloud. We had been running our applications on a fixed number of servers which we spent weeks to procure and configure, mostly by hand with little to no documentation or automation. We gave these servers cutesy names like they’re pets. Deployments often involved scheduled downtime. Servers were too often in a single geographic area.

Not surprisingly, a first endeavor to deploy in the cloud often involves as little adaptation as possible — we’re doing it a lot like we used to “back on Earth” by hand configuring servers, imaging them, and launching VM’s from those images. So we’re technically “in the cloud”, and there are some powerful benefits already, but we’re only halfway to solving our problem.

In the cloud we’re deploying on to VM’s or containers which may launch and terminate in as little as a few minutes. And they might be behind a load balancer and firewall that was just created a moment prior with an API call.  Scaling is elastic, measured, and operates automatically based on real-time conditions.

Launching an application into the cloud is a lot like launching a satellite into space.

Your app is running on instances on the east coast, then it’s on the west. Next time you look it might be in Asia. Getting information from it is like telemetry.  It’s a helpful mindset to consider servers in the cloud as being inaccessible remote locations because it reminds you to plan ahead what stats, data, and logs you want to get back from it and build an automated means of transmitting that data to a central place. You can’t wait until something goes wrong and then expect to ssh into the server and look around, it may well be terminated by then. In fact, just to remind you how short lived they are, we don’t even call them “servers” anymore, they’re “instances”.

Before cloud architectures, the assumption was the server was a “going concern”. It was going to be there and continue to operate unless something catastrophic happened. With cloud, this assumption is turned on its head. For reasons like auto-scaling, health check failure, or even getting your hourly price out bid in the spot market, your instance could be terminated in a moments notice. All of this means we now have to build our apps in a way that is tolerant to sudden shutdown and can resume by another node. We can’t even rely on short term storing of any state in memory or on disk.  Consider them literally as they are called, “compute instances”.

We have to have a strategy for health checking our applications. Sure we’ve had monitoring before, but now our infrastructure will automatically terminate any instances that fail to meet the health check requirements — and if you’re not careful, can terminate your entire cluster for something that might have been a momentary hiccup, leaving you completely down for 5 to 10 minutes until new VMs boot up.

These are among the many fascinating DevOps problems you will need to think through and solve as you devise your own way of deploying and scaling apps in the cloud.

All of a sudden you realize you’re building your own platform.

A more complete picture is that DevOps isn’t just automating deployments, it’s building or updating applications to 12 factor apps with scaling and ephemerality in mind, which implement your established health monitoring and logging standards, and building a pipeline and set of practices for deploying and operating those apps.  We don’t just get that out of the box with our IaaS provider.  And so I think a lot of the work which is currently done out of necessity under the umbrella of DevOps will increasingly be provided by platforms that sit a layer on top of IaaS.  Knowing how much work goes into building all of this on your own, I believe that PaaS is the future, for the same kinds of reasons why IaaS has value over building your own data center.

We need DevOps because we need the cloud, and the reality is that DevOps is just necessary to deploy and operate modern apps in the cloud, weather you are doing all of it yourself or leveraging a platform for some portion of these functions.

UX So Bad It Just Might Kill Someone: Audible’s New Clips Feature

Even with the recent rise in focus on user experience (UX) testing, I think the practice in general is frightfully under invested in. I don’t have unreasonably high expectations of products, but sometimes the UX is so bad that it makes me literally shout out loud in anger at the company who made it. Such was the case from my experience with the new Audible app this morning. I have listened to hundreds of books with Audible over the past several years. I’ve recommended Audible along with the practice of using your commute to learn to anyone who’ll listen. Lately I’ve been listening mostly to podcasts, so I didn’t notice the recent changes to the Audible app until this morning.

They’ve replaced “Bookmarks” with “Clips”. Sounds harmless. In fact, the clip feature can be useful, if say I’m listening to a book while lying back on a beach chair by the ocean where I can casually take the time to mark the beginning and end point of my clip and edit it to make sure I remove any unintended words. But like most people, I listen to audio books mainly in my car, and you can’t use this feature while driving without seriously risking your life. I can’t bother with marking the beginning and end of my clip you *$$@#!#@$#  $#!@%, I’m driving! I tried to use a clip like a bookmark but it resulted in making a large clip from the beginning of the book to the point where I wanted to bookmark.

It’s labeled in the app “Clips and Bookmarks”, seemingly implying think the plain old bookmark functionality is still there somewhere if I can just find it (it’s not).  It was dangerous enough using bookmarks while driving, but clipping is completely out of the question. It’s a shame because I really value my bookmarks.  But this feature is just too dangerous to use while driving.  I guess I will have to make sure the important points sink in the first time.  I just expect more from large companies like Audible (Amazon), in part because I know it’s likely that there’s a team of people who work there whose focus is UX.

I know, I’m not supposed to be using my phone while driving in the first place.  But I’m not sure how realistic that is as a hard rule.  This isn’t exactly texting.  I think certain apps like music, audio books, and GPS navigation, are meant to be used in a car and it’s unrealistic to expect zero user interaction while driving.  If these apps are made with the expectation of light use while driving, they should be no more difficult to use than operating the built in car radio or air conditioning.

2015: A year of gratitude journaling

In late 2014, I started keeping a gratitude journal.  I have an appointment on my calendar every weekday at 12:30 pm to remind me to do this.  I try to take a few minutes, usually while eating lunch, to journal things I’m grateful for that day.  I do this by emailing myself with the subject line “gratitude” and saving them into a folder.  In 2015 I had a total of 115 entries.  Here are the most frequently used words in those entries, in the form of a word cloud.
2015-gratitude-word-cloudI’ve read several books which encouraged the practice of gratitude journaling, but it was Oprah Winfrey’s 2014 book “What I Know For Sure”, which convinced me. I think her words say it best.

“Sometimes we get so focused on the difficulty of our climb that we lose sight of being grateful for simply having a mountain to climb. I know for sure that appreciating whatever shows up for you in life changes your whole world. You radiate and generate more goodness for yourself when you are aware of all you have and not focusing on your have nots. It isn’t easy, but it’s when you feel least thankful when you are most in need of what gratitude can give you — Perspective. Gratitude can transform any situation. It alters your vibration, moving you from negative energy to positive. It is the quickest, easiest, most powerful way to effect change in your life. This I know for sure. Here’s the gift of gratitude. In order to feel it, your ego has to take a back seat. What shows up in its place is greater compassion and understanding. Instead of being frustrated, you choose appreciation. And the more grateful you become, the more you have to be grateful for.”

If I could hope for any one thing for my daughters, it’s that they will someday choose to set aside time on a regular basis to acknowledge the things they’re grateful for. The hope of that literally brings a tear to my eye. I know that this practice has helped me appreciate life more in the present moment, and helps shift my thinking towards possibilities and opportunities in the future.

Why I Went Back To Being a Software Developer

When I was a kid, I was told that a midlife crisis was freaking out because you suddenly realized your life is half over, which results in buying a red sports car and running away with a new 20 year old girlfriend. While I’m not sure if 38 qualifies for midlife, I’ve certainly become acquainted with the concept, and I think a more accurate depiction of midlife crisis for some, is finding yourself nearly half way through your career and still not knowing exactly what you want to be “when you grow up”.

I started as a software developer and found initial success there but I really wanted to be on the business side.  So I founded my own software company which pivoted into an online retailer, then moved on to senior management roles in large established retail businesses running e-commerce and digital marketing. Now I’m going back to being an individual contributor in a software development role.  I feel like I owe at least myself a well articulated explanation for taking several steps back on the org chart, after investing nearly a decade on the business side of an e-commerce career path.

First, why I chose to move away from the business side of e-commerce.

Retail e-commerce is a largely undifferentiated space.

Prior to the Internet, retail stores had to compete mainly with other physically nearby stores. Many found success largely due to this limitation on the number of competitors.  Online the dynamic becomes, why should someone buy from you when they have hundreds of other options, most of them offering better pricing, better delivery times, more selection, and better customer service? It’s more important than ever to have a unique selling proposition.  You can’t just show up offering the same thing for sale that everyone else is selling.  You actually have to provide value (cheaper, faster, friendlier, more selection, unique selection, more helpful) to be successful.  That may sound obvious but nearly all online retailers bring nothing unique to the table.

Online retail is increasingly becoming a winner take all proposition, dominated by a handful of players.  The top fifty retailers in the U.S. control 75% of all e-commerce volume in the country, and the top four retailers (Amazon, Apple, Walmart, and Staples) control 40% of all e-commerce.  And as lines continue to blur between online and offline, this pattern will not be unique to retail.

The lesson for me is that if your business isn’t uniquely providing value, it is just as ephemeral as the EC2 instances it’s hosted on.

When you’re on the business side of e-commerce, the expectation from senior management is often to provide transformative results while only having the autonomy to make incremental improvements.  There’s this unspoken notion that you can just sprinkle e-commerce pixie dust on the website and extract your share of Internet riches.  You know, just find those perfect keywords, do that hipster magic in social media, and split test the color of your Add to Cart button until you find the optimal shade of green.  But all the best practices in the world are not going to move the needle if you’re not competitive on price, selection, or convenience.

Often there’s an over-emphasis on marketing to “get traffic” and “make that traffic convert”. Marketing’s role is to position ads to be in front of as many top quality purchase prospects as possible at the most effective cost of doing so, but it can’t get people to buy. That’s the job of the core business. And if the business you’re promoting isn’t better than its competitors in some meaningful way, you’re just not setup for success.

So why move back into tech?

Because the Internet has made the world so much smaller, the resulting winner take all pattern makes the company you work for really important, no matter what your role is.  At many of the best companies, engineering is valued above marketing, and is a better bet to finding yourself on a winning team.

I started to think about all the companies I most respect, and the pattern was they were all innovators in technology.  I realized that what I loved was working with the Internet, not retail, or selling the exact same thing everybody else is selling, or putting a SKU into a box and shipping it.

So does this mean I intend to be a geek for the rest of my days and have forever left the business side?  Not exactly.  As business and technology become more tightly coupled, I think this path will lead to different business opportunities – better ones.

Lastly, in regard to taking a step back from management, being a leader and being a manager are not necessarily the same thing.  Being a leader does not require a title, and sometimes you can have more influence as an individual contributor than you can as a manager.

Store Locator – The Most Underutilized Asset In Online Retail

Want a quick way to increase your online sales?  If you’re an omni-channel retailer, there’s probably something simple you can do right away and may see a 10% bump as a result.

Your store locator probably gets in the neighborhood of a quarter of your site traffic.  And nearly half of these store locator visitors bounce without exploring your site any further.  What a shame.

When a customer uses your store locator to look up a store, it’s a strong indicator of intent.

They’re not looking up your location for the fun of it, they’re doing it because they’ve already researched the product or service and are further down the funnel, ready to take action.  They’ve half made up their mind to buy from you and now they just need to get your address or your phone number so they can drive to your store.  Traditional retail folks know that once you’re physically in the store the conversion rate is very high, typically 30% or more, which baffles the minds of strictly online guys, and is a phenomenon worthy of its own research.  So that’s great but if you’re reading this you’re the e-commerce guy whose job is to grow the online business, at least to the point where it isn’t so damn pathetic as a percentage of the overall business’s revenue.

Here’s an easy score – put a call to action on your store locator pages which encourages customers to at least look at the online store before closing their browsers.  Do you have a sale running with a promo code on your store?  Convenience features like reserving items or pick up in store?  Highlight this from the store locator with a link to online shopping!  If you have any sort of service or consultation component to your business, by all means accept online appointment scheduling.  Customers browsing the store locator are primed and ready to convert.  Do some simple things to get more of them to convert online where you can track it (and before they change their minds).  These are typically some of the easiest changes you’ll make as we’re mainly talking about some creative and a hyperlink, and I’ll bet you can see results that outperform that last big feature you spent 4 months of development on.

So if your store locator does nothing more than give store location and contact information, for heavens sake do something to encourage them to maybe shop online before just closing their browsers.  You’ll be glad you did.  And while you’re at it, if you’re not making bi-weekly payments on your mortgage, sign up for that today too.  Seriously, you can take years off that thing and you get paid every two weeks anyway.  There, that’s two solid pieces of financial advice.  You’re welcome.

Branded Search – Thine Friend, Thine Enemy

Once upon a time I didn’t know what “branded search” meant.  I would have assumed it referred to people searching for a particular brand of product like Nike they could buy at a retailer like Amazon; searches like “nike shoes”, not searches like “Amazon”.  Of course, branded search actually refers to your own brand.  “Nike shoes” is a non-branded term, unless you’re Nike.  Who knew.  A bit of a misleading label in my opinion.

The reason I didn’t know what branded search meant was because up to that point my e-commerce experience was from the perspective of a startup.  When you’re doing less than $10 million in revenue, you generally don’t have a “brand”, and certainly don’t think of yourself that way.  Do you buy Google searches for your own site’s name?  Of course you do.  They convert great, don’t cost a lot, and there are few people searching for you anyway.  Life as a non-brand often means doing battle in Google for every order.  If you were to turn off your advertising, your sales would immediately fall to near zero.

For startups and small online businesses, a pie chart of what traffic sources produce orders, looks something like this:

80% Non-Branded Traffic, 20% Branded Traffic

Now that I manage the online business for a very large, very old, publicly traded company with a large number of physical stores across the country, of course this is where I become acquainted with the term “branded search”.  From the perspective of a startup, you always look at the big brands and wonder things like what their conversion rates are like, how they get their traffic and why they’re so successful.  Well kids, having seen what’s behind this curtain I can tell you there’s nothing to be impressed with.

If you ever looked at the Alexa rating of a huge brick and mortar incumbent in your industry and wondered how they get all that traffic, allow me to demystify it for you – almost all their traffic comes effortlessly as a result of being an established brand with physical stores that people drive by and shop at every day.

They don’t know something you don’t.  They’re not raking in orders via social media.  They aren’t crazy good at paid search, and their conversion rate is probably worse than yours.  They get this traffic automatically just because of who they are.  Even the natural search traffic they get is by far more as a result of Google viewing them as a brand rather than the content being fantastic or really well tuned for SEO.  For large multichannel retailers, the pie chart of the traffic sources that produce orders generally looks something like this:

20% Non-Branded Traffic, 80% Branded Traffic

The problem is most big brands aren’t looking at their world that way.  They might list orders by traffic source, which will make it appear that Natural Search and Paid Search are very large buckets of where their business comes from.

When people hear “paid search”, they generally think of all the searches for products, not the searches for your own company’s brand name.  I think many would be shocked to learn just how much of that is really people just looking you up in the white pages.

If you pick up the phone book and go right to the white pages to find “Bob the Plumber”, this means you’ve pretty much already decided to do business with Bob.  Obviously once you connect your call there is going go be a high conversion rate.  If on the other hand you go to the yellow pages and look in the plumbers section, you may call several plumbers before deciding to do business with one.  However if you’re Bob the plumber and that customer didn’t know you, they may have only found out about you from your listing in the yellow pages.  It’s expensive but it can be very effective, which is why you pay for it.

Online, big brands get most of their orders from the white pages (branded search), which is cheap and converts very well.  They are also spending a boat load of money to be in the yellow pages but often don’t realize how much they’re spending to acquire customers there because the paid search dollars are actually spending on both the yellow pages and a double listing in the white pages (buying your own brand name in paid search).  The results are often reported mixed together since it is all paid search spend.  The yellow pages are really expensive and not performing super great but the double listing in the white pages delivers and more than makes up the difference.  Typically marketing reports lump all paid search (branded and non-branded) together so it appears as though paid search is really effective and is a huge driver of your business.  Be mindful that this is what Google wants you to believe (keeps you sending huge gobs of money their way), and also what your agency wants you to believe (it makes their job a lot easier and keeps money flowing their way too).

I consider people searching for your core brand equivalent to direct type-in traffic. Meaning someone searching Google for “kohls” is equivalent to typing kohls.com directly into your browser.  Same intention.  Many people forget to put the .com and the browser performs a search, other people just search for everything.  But the intention behind the act is the same.  So when I say “branded search” I’m referring to both, essentially people who were already looking for you and intended to come straight to you.  And yes I include any branded variants (“kohls clothing”) as well as core brand (“kohls”).  Just because people may believe they’ll be able to navigate your site better using Google doesn’t mean the intention wasn’t to go directly to your site.

For many multichannel retailers, if they had to rely on profitably acquiring new online customers who weren’t already looking for them, they would go bankrupt.  Many multichannel retailers online business doesn’t have real P&L accountability, and many aren’t even using a shadow P&L.  They may fancy themselves as profitable largely because they don’t understand branded traffic.  This seems funny to me that the big brands don’t understand the pros and cons of branded search in online business.  I think it’s because when you experience life without the benefit of branded search, you recognize it right away when you see it, and you can appreciate the benefit, the “unfair advantage” if you will, that it is, and how that built in advantage can blind you from the true effectiveness of the online business.

Why is branded search your enemy?  It’s not, however misunderstanding the big picture can be.  What happens is that the branded search subsidizes the non-branded search.  This makes you misjudge the effectiveness of your paid search campaigns which can cause you to both loose profit and fail to raise revenue.  You loose profit because you mistakenly believe your paid search campaign to be profitable while in reality you may be handing over the lion’s share of your hard earned gross margin to Google, who is more than happy to take it.  It’s a different kind of stupid tax for the rich.  You may loose revenue opportunity because you aren’t able to truly tap into this traffic source to grow the business because you’re not as focused as you should be on making your non-branded campaigns more efficient.

Of course ultimately branded search is how you win.  But the wining is not in the buying of the branded search, it’s in the creating of happy customers that want to buy from you again and cause those people to search for you.  Just don’t confuse the two.

If no-one is searching on your brand and your business is massively dependent on new customer acquisition, you might as well consider yourself an extended employee of Google because all the live long day you’re just working hard to put money in their pockets.  Knowing how to profitably acquire customers in Google is obviously important, but in order to win long term, focus your resources on getting that customer to buy from you again.