We need a new website.
We hear these words from our clients a lot. And while it’s most often very true—they do, in fact, need a new website—it’s only part of the truth. Because websites don’t exist in isolation. They’re part of a digital ecosystem of interrelated tools that, designed properly, work together to solve business goals. And the website itself is a collection of tools and technologies that must work seamlessly together, so our ever-evolving approach to website engineering must be strategic, forward-thinking, and robust.
Webpack, React, and WordPress are part of a technology bundle we’ve created to help us modernize our development stack, allowing us to build these robust websites and ecosystems more effectively and efficiently. Read on for how we got here—and why we consider this collection of tools the right foundation for the realities of today.
Asset and build management in WordPress have become a complicated and interesting challenge in recent years, mostly owing to JavaScript’s rapid growth and subsequent effect on how we manage frontend dependencies and create optimized bundles of our assets. In this article, we’ll explore the current iteration of our build system and how we architect our JavaScript and CSS to achieve this set of goals:
- A unified build system that avoids code duplication and maximizes resource sharing
- High performance, optimal delivery of client side code
- The ability to inject apps into the JavaScript bundles as on-demand chunks that share vendors and utilities with the rest of the codebase
- An enjoyable and fluent developer experience that brings the best of current client side development to the WordPress workflows we love
But before we get into some code samples and all the riveting details, a little bit of history. If you want to skip the nostalgia fest, jump on down to “Let’s Work it Out.”
Which File Was That Again?
When I joined Modern Tribe—eight years ago now—frontend life in WordPress still consisted mostly of monolithic files that were usually written using heavy servings of jQuery and plain CSS or Sass for the styles. Global variables were mostly used to shuttle data/methods around, and things were generally difficult to maintain and organize at scale. For us, extending WordPress was based on the common approach of a plugin for each larger feature, meaning a big system could end up with one or two JavaScript files for each plugin, plus the main theme scripts, and if doing child themes, additional files for those.
This obviously led to a lot of code duplication if not managed well and real difficulties managing vendor dependencies. For example, seven files rely on Select2, but three at an older version, and oh, yes—those third-party plugins we dropped in, well two of those are also loading up other versions of it for their use. Then we got to the minifying of all these files effectively and attempting to reduce the amount of network requests for this peppering of fragments from 23 directories. This saw many optimization add-ons using PHP to intercept all enqueued scripts and concatenating them, but this was prone to issues with order of execution and other problems.
We had to find a better way to manage all of this.
The Birth of SquareOne
Our solution to these and other issues we were facing was the creation of our internal framework, called SquareOne. It consists of a range of methodologies and patterns we have developed—and continue to evolve—to vastly improve our day-to-day work with WordPress. Its high-level goal is to provide for all of our projects a consistent and reliable codebase that takes care of common tasks and shared utilities. It’s a modular system that scales smoothly from smaller projects all the way up to demanding, enterprise-level WordPress applications, all with minimal friction.
Regarding frontend development and our build system, SquareOne was initially based on Grunt and used Bower for vendor management more than six years ago. We used Sass for styles and still managed multiple feature-based plugins, each of which had its own dedicated Grunt/Bower subsystem if needed. There wasn’t much gain over the original issues, so we continued to explore ways to unify the codebase, vendor management, and its frontend build systems.
Rapid Change Within the JavaScript Ecosystem
At this point in time (five years ago) the JavaScript ecosystem was about to get its biggest shake up yet—the release of ECMAScript 2015. This, combined with the wave of server-side/script-level JavaScript execution that was taking hold, led to a language, which had enjoyed years of bemused scorn from many developers, becoming the most popular “new” language to work in, while still enjoying the bemused scorn of many developers. That said, there was also a massive wave of fresh interest and development in it.
With the solid traction NPM was gaining, thousands of developers started to share their frameworks, libraries, and utilities in an easy-to-consume system that spelled the death of Bower and the like, at least for all but the masochistic. A new paradigm had settled in quickly and was prepared to fill our workflows, servers, browsers, and hard disks with its enthusiasm.
These rapid changes also brought forth some of the most influential and heavily used frameworks for frontend development—systems such as Angular, Vue, or our framework of choice, React. Why “influential”? Because the purpose they filled (building complex applications that run in the browser) caused the ecosystem to explode even further. This occurred both in plugins/libraries that enhance their core functionality, as well as build systems that could effectively manage the complex dependency trees and resultant asset bundles that these new applications required.
Webpack, What a Lovely Headache
Cheeky headings aside, a new module bundler emerged as the leader in a family of systems whose purpose was to handle this new level of complexity more gracefully at the compilation level. At Modern Tribe, we dove into version one of Webpack and quickly incorporated it into our existing Grunt-based build systems to handle our JavaScript bundling. At this time, like much of the community, we had transitioned into using Babel to enable the usage of all this new JavaScript syntax we had available in legacy browsers. This organically caused a transition from reliance on jQuery to using the new vanilla equivalents as well.
Now, the light headache associated with this transition generally revolved around quirks in the young bundler combined with its confusing (at that time) documentation. More complex and nonstandard builds required some trial and error, plus lots of digging in issue trackers to find your way around. But the result was still love. While version one didn’t have a lot of the great features we enjoy today, the base was there to allow us to start organizing our JavaScript into concise modules and directories, import dependencies in a snap, and create much more optimized asset bundles with ease. We finally had a flexible workflow that handled any of our needs with only a minor increase in temporal blood pressure.
Hello Unity, My Old Friend
We now had a much better build flow in place. We had also made some other important steps in our evolution of SquareOne:
- We implemented a single core theme and plugin, allowing our architecture to function more easily as one seamless application rather than a collection of discrete, unrelated plugins, each with their own autoloaders, build processes, and dependency trees. Additionally, this paradigm allows us to use a single Service Container to handle any dependency injection or class composition needs, providing a unified system for transporting dependencies among the various pieces of the application.
- We moved to PostCSS to streamline our CSS work and gain more granular control over our bundle output.
- We began to orchestrate the admin and theme JavaScript/PostCSS bundles from one unified build system, which was based in the root directory of the WordPress install.
- We began using Webpack code splitting to break up our JavaScript code into smaller chunks and only load required code as needed across the admin and theme.
We had achieved our goal! We had one unified dependency management and asset bundler at the root of the project, able to serve up all we needed for both the WordPress admin and the theme. We could share JavaScript libraries, modules, and utilities across these contexts, as well as all those PostCSS variables, mixins, etc. that we had employed.
Story over, right?
Nah, let’s make this more complex. Somebody somewhere said, “Hey, we should build two of those admin screens and a few sections on the frontend of this PHP-served WordPress app in React.”
Challenge accepted.
The Best of Both Worlds?
We had been building React apps and integrating them into WordPress quite frequently over the last four years. We had also built Node-served SaaS platforms, like Loxi, utilizing full server-side rendering. There were also pure React frontends like our own tri.be website, or our own custom page builder for WordPress, Panel Builder. But the hybrid injections of React to drive a portion of a standard LAMP-served WordPress build were proving more difficult to engineer than the ease of building a singular React-based app on a Node-hosted platform or the like.
Over the first few years, we found ourselves reintroducing the same issue we had worked so hard to resolve with our unified build system. Sprinkling a couple of React apps throughout our main vanilla JavaScript bundle (following the standard approach to building one) left us with a root level build for the main JavaScript/CSS bundles and a separate dependency/build system for each React app. A few obvious issues were introduced by this:
- Babel polyfill can’t be used twice, so the React apps would have to use runtime
- High possibility of duplicated dependencies in the bundles
- Difficulty sharing custom utilities in a concise way across builds
- Back to multiple build system and dependency version management
You may be thinking, “Why are you using React to drive portions of the WordPress frontend/admin? Just build it all in plain JS.” This is a reasonable suggestion for many cases, and we do follow that rule when we can. But we have found enough use cases where React was the right tool for the job on some of an otherwise vanilla JS-based system.
Now we had to find an effective way to manage this all from one truly unified build flow.
Let’s Work It Out
Well, that’s more than enough background for those of you who slogged through, let’s get to the current build system we use to achieve the goals we set out at the beginning of this article. This will be a combination of a few code samples, along with links to a public repo. Speaking of which, you can load up this GitHub link in a new tab and refer to the code as the principles are being discussed.
Before we start, a few notes on this system. While pathways to source and build files correlate to the directory structure we use, this is very malleable. You are not required to align with our paradigm of a single “core” plugin and theme, and all paths can be easily adjusted to whatever structure you like to use. It’s also possible you don’t care about having an admin or theme bundle generated, depending on what you want to do. You can simply ignore the unneeded aspects, or if you like, clean out those portions from the build, though they don’t add any overhead until they are in use.
Let’s get to it.
Overview
As mentioned, our build system lives at the root of the WordPress install. This allows us to easily manage bundles anywhere across the system from one central place. We moved from Grunt to Gulp a while ago, and we still require it on top of Webpack to handle some of the other tasks we need to accomplish. These could have been scripted in other ways, but the convenience of the Gulp ecosystem and availability of many tools made it the right fit for us.
If you haven’t already, be sure to load up the example repository in another tab now.
Package.json
Our package.json is very hefty and contains all the tools we use out of the box across our projects. It also contains granular paths (used in the Gulp/Webpack config files) for each aspect of the output files across the pcss and JS directories. This is the place where you would adjust those paths to suit your theme and/or plugin structure. Also note that we prefix all variables with an underscore. Lastly, we set our browserslist array here, just under the variables stack, so adjust accordingly.
Gulp Configuration
Our Gulpfile handles a lot of tasks. These are broken up by task type into the gulp_tasks directory located at root and then imported into the main Gulpfile using require-dir. They are then wired up using the registerTasks function below the gulpTasks array. This allows us to scale to a very large set of tasks while keeping them clean and organized.
Note that we also run a Browsersync server for live reloading and device sync. This is run using the dev task, and then our only other task in this example is dist, which bundles both the scripts and styles. This can be broken up into sub-tasks that fit the flow you desire easily, and we do so on a project-by-project basis.
Webpack Configuration
Our Webpack config is also broken up into manageable files, with partials located in the webpack directory found at root. Since we want to be able to run a variety of different bundles with minimal code duplication, we also use this technique to break out common aspects of the various tasks and merge those into the unique parts of the flows using webpack-merge.
You may also note that the root webpack-config relies on node env to load the correct file for the task being run. This way, not only can we target dev and production bundles for our admin and theme tasks, but we can also use arbitrary NODE_ENV keys to run our React app chunks in hmr-enabled dev mode during work on them. More on that in the React section later.
webpack-config.js
common.js
This is our common tasks file. Not much to note here, other than special handling for a few of the libraries we use that require that. This is merged into all our theme and admin tasks.
themedev.js
This file contains our theme development tasks. We merge in common, setup our entry and output paths and naming conventions for the chunks, and enable webpack bundle analyzer to keep track of our bundle size.
vendor.js
This sample is our vendor arrays that declare third-party libraries used in our JavaScript. These are used by the admin and theme Webpack configs, both dev and production. Any time you use a new vendor that should go to the vendor chunk for those respective bundles, you add it to the array here. This separates the JavaScript that rarely changes from the chunks that do change all the time, and it’s therefore more efficient when forcing cache busting on browsers as new code is pushed up.
JavaScript
We aren’t going to dig deep into the JavaScript config we use, but will pull up the core initializer called on document ready. For both the theme and admin bundles, we have a similar file. We import a few global config handlers, and then our main entry points for sub systems, e.g. handlers for singles, a navigation system, etc.
We also have a system in place to inject polyfills as chunks based on browser detection of features, as can be seen in the setupEnvironment method below. This means polyfills only inject based on need, and all subsequent code is deferred until the chunk-loading promise is resolved.
Any other code that should only load based on a context should also use chunking here. Observe the commented out example React app loading as a chunk in the sample below. You can use element exists, a route check, or some other method like a body class exists check to do your own conditional chunk injection for any other portion of the imported sub modules as needed.
React
As with the JavaScript above, we won’t go in-depth on the React code approach in use here. You can dig through the SquareOne repo if you care to see how we organize our code, but the basic gist is that we use redux, redux-sagas for async code, and reselect for our selectors. We break up these elements into sub reducers, ducks, and sagas per app screen/route.
The important part here is noting how we use chunks in the above entry point example to inject React apps as chunks and share our vendors across both the main JavaScript bundles and the React chunks. This means we can dev React apps in Hot Module Replacement mode on a Webpack server as needed, and then our dev and prod bundles can inject the apps as chunks if the app root nodes are found on the page.
You can configure your React system however you like. We just include our entry point here for the sake of completeness.
WordPress Config
There is a little bit of configuration we will need to do in order to switch into Hot Module Replacement mode when we are working on React apps, as opposed to when they are just loading as chunks in the main bundles.
First we need to define a constant in our local-config.php (or however you handle WP dev config) like so:
Then we setup this enqueue to run when that flag is true in the same file we use to enqueue our other scripts.
This points to our Webpack dev server output file when we run our NPM script that kicks in the React app Webpack configuration, e.g. app:example in our package.json above. Combined with the fact that our chunk loading for our React app won’t execute if we are in HMR-enabled dev mode, we can avoid double loading of that code when working on any of the React apps.
One last note, we have one other issue to handle. When we aren’t in HMR mode but have a React app or two in the mix and we wish to develop on our main theme or admin JavaScript bundles, every watch task run would cause these heavy app bundles to be rebuilt. This would make the dev experience very sluggish. To avoid this, we use a Webpack plugin called if-def loader, and have our React app chunks inside a comment that the plugin reads to decide whether or not to include some code during a build task. You’ll see the core.js example above contains the comment around the example app chunk. We then run the loader on our gulp watch task like this:
Wrapping Up
So there we have it, a way to manage assets for the client side in highly scalable WordPress apps, which also allows for blending frontend methodologies to your needs easily. Whether you just want some clean vanilla JavaScript to run the frontend of your site, or you need to take it further and start sharing across contexts and framework types, you can scale and blend this brew to whatever custom approach you require.
Of course, engineering systems like this are not for everyone. While the tools are very powerful once you get comfortable with them, the reality is that they can be very time-consuming and at times brittle when managing a build stack like this. As such, ways to simplify this kind of tooling overhead are on many a dev’s list, and we are seeing many promising solutions cropping up, with SnowPack currently at the forefront. As these approaches mature, you can bet that we will be weighing them for merging into our systems. But for now, that wonderful ringing in our ears is still saying: Webpack and its ecosystem are the best way to build a modern development stack.