Drew PowersEngineering @ Manifold

Meet @pika/pack, your new favorite bundler for npm. It handles Babel-ification, setting up your package.json, entry file, and even comes with a built-in publishing command to make versioning dead-simple.

Is @pika/pack for me?

This is a question that comes up with every new technology. To help with that, I’ve made a handy chart:

Should I use @pika/pack to bundle my library?

Are you shipping your library to npm?Then…
🚫 NoProbably not
YesYou should use @pika/pack

🚀 Shipping with @pika/pack

Prepping

Whether you have an existing npm library or a new one, this is all you need for preparation

  1. Have your entry file live in src/index.js or src/index.ts. The entry file determines everything that will get bundled. For simplicity, this isn’t configurable. But consistency is a good thing! Trust us.
  2. If your library is written in JavaScript, it needs to be in ESM (importing non-ESM external libraries is OK though). Sure, this may be a pain if your entire codebase is in CJS, but 👩‍🚀 ESM is the future!

If your codebase is in TypeScript, you’re all set for #2 (@pika/pack 💙s TS).

Configuration

Configuration for npm happens in—where else?—package.json. Setup between TypeScript and JavaScript is pretty similar, but with minor differences.

🐣 Basic setup

Run the following in a terminal:

npm i -save-dev @pika/pack

And add the following lines to your .gitignore (you’ll need this later):

pkg

If your code is primarily running in browsers:

npm i --save-dev @pika/plugin-build-web pika-plugin-minify

…or if your code is primarily running in Node.js:

npm i --save-dev @pika/plugin-build-node

If your code runs in both… install both! Though most libraries are primarily one or the other, @pika/pack has no problem shipping universal code.

Add an npm script to package.json that we’ll use later:

  "scripts": {
    "build": "pack build"
  }

We’re not done yet, but we’re very very close. The last step will depend on whether your package is written in TypeScript or JavaScript.

🤖 TypeScript

In a terminal, run

npm i --save-dev @pika/plugin-ts-standard-pkg

Add the following to your package.json

  "@pika/pack": {
    "pipeline": [
      ["@pika/plugin-ts-standard-pkg"],
      // ["@pika/plugin-build-web"],  /* Use this for browsers */
      // ["pika-plugin-minify"]       /* also use this for browsers */
      // ["@pika/plugin-build-node"], /* Use this for Node.js */
    ]
  },

💁Tip @pika/plugin-ts-standard-pkg uses your tsconfig.json, so it’ll inherit things like includes and excludes from that. If you find it bundling things you didn’t intend for it, then try editing tsconfig.json. Less config is more better!

🐴 JavaScript

In a terminal, run

npm i --save-dev @pika/plugin-standard-pkg @pika/plugin-build-types

Add the following to your package.json

  "@pika/pack": {
    "pipeline": [
      ["@pika/plugin-standard-pkg", {"exclude": ["__tests__/**/*"]}],
      // ["@pika/plugin-build-web"],  /* Use this for browsers */
      // ["pika-plugin-minify"]       /* also use this for browsers */
      // ["@pika/plugin-build-node"], /* Use this for Node.js */
      ["@pika/plugin-build-types"],
    ]
  },

Configure exclude to ignore things like your tests, or any files that definitely aren’t a part of your bundle. It’s an array, so you can ignore multiple patterns.

Did you happen to notice that @pika/plugin-build-types? That’s right—@pika/pack will ship types for your project even if you didn’t write them yourself. 😱 Now you can finally close that Issue you’ve been ignoring for months!

🧺 Optional: extra files / folders

Does your npm library have a binstub, or extra assets that are crucial, that aren’t part of the core codebase? @pika/pack has got you covered!

In a terminal, run

npm i --save-dev @pika/plugin-copy-assets

And specify which folders you’d like to ship to npm in package.json with the files option:

  "@pika/pack": {
    "pipeline": [
      // …
      ["@pika/plugin-copy-assets", { "files": ["assets/", "bin/"] }]
    ]
  }

📦 Building

Now it’s time to ✨make some magic. Run:

npm run build

From that build script we added above. It should spit out a pkg folder in your project root. Inside you’ll find:

pkg/
├── dist-src/
├── dist-node/ *
├── dist-web/ *
├── README.md
└── package.json

* may or may not exist, depending on your build targets

Let’s break down a few very clever things @pika/pack did:

  • It moved everything into pkg, which makes it much easier than managing includes/excludes in the root. It’s like dist, but for npm! Very clever.
  • It made a new package.json that’s hand-curated just for npm. Ever shipped a broken version to npm because of a package.json misconfiguration (if you haven’t, you’re a legend)? No longer an issue!
  • That new package.json still has all your old entries, settings, and configs. But it added things like module and main which make sure everything is wired up as it should be.
  • It copied your README!

At this pont, your package should be built, and ready to ship to npm. Although I really want to geek out over what a big deal this is (not only for your library, but the future of the ecosystem) I’ll come back to that and instead and finish the tutorial part by talking about @pika/pack’s publishing capability (stay tuned for some knowledge though).

🚀 Publishing

You probably haven’t been timing yourself up to now, but if all went smoothly, it has probably been only a matter of minutes that you’ve been able to build a high-quality npm package that has offloaded all the gruntwork onto @pika/pack. But we’re not done till it’s live, right? Let’s go live!

@pika/pack has a pack publish command that is really similar to npm publish, but has a few nice things added on top. Everything here you can 100% do with npm publish, only that would take several more config flags and a bit of manual work on the side. Every bit helps.

To make this easier, let’s add another npm script to package.json:

  "scripts": {
    "build": "pack build",
    "prepublish": "npm run build",
    "publish": "pack publish"
  }

💁Tip In npm scripts, you can automatically set any task to run before another just by adding pre to the beginning (or after with post). By adding prepublish, we’ll automatically do a fresh build right before publishing—very important so we don’t actually release stale versions.

After that’s added, run npm run publish (the “run” in the middle is very important—without it we’re not using @pika/pack). You’ll get the following prompt:

Select semver version or specify new version (use arrow keys) > patch / major / minor / prepatch / preminor / premajor / prerelease

The first question is: what kind of release is this? You could read npm’s semver notes, or follow my personal rules of thumb:

TypeNotes
Majorlol refactored everything
Minormany changes that probably won’t break your stuff but 🤷‍♂️
Patchfixed a bug you hopefully haven’t noticed yet
Pre-major/minor/patchjuuuust want to make sure before I bump this unchangeable version forever
Prerelease😎 my little secret

The next question it’ll ask is what tag you’d like to use. This is one of the most underused, powerful features of npm. This applies when releasing a pre-* version:

Select semver increment or specify new version _prerelease_. How should this pre-release version be tagged on npm?

By default, @latest is your “master” tag. Everytime someone npm i your-package, they’re installing your @latest tag. So in order to release beta versions, or have alternate versions, or literally whatever you want, this is your chance. You may have even dived down some rabbit hole in GitHub issues, and the one solution that worked was installing a beta version via npm i package@next.

Tags are whatever you want to make them, and therein lies their power. Other than latest, tags have no meaning. next, beta, and canary can all mean the same thing—it’s up to your standards and documentation to define them. With tags you can ship as many different versions of your package as you want—all within the same namespace. Cool!

Finish the prompts, and you’ll be greeted with a glorious sight:

🎉 @manifold/swagger-to-ts v1.1.2-0 was published! You can see it at: https://unpkg.com/@manifoldco/swagger-to-ts@1.1.2-0/

@pika/pack took the liberty of checking out a tag in Git for you, running tests, and checking everything is good… and that was just for a throwaway prerelease!

💁 Tip Want to skip all the testing and checks, and just get it live? Add the --yolo flag to the publish npm script above. It couldn’t be better named.

Conclusion (kinda)

Above I stopped myself from geeking out over what a big deal @pika/pack is. Yes, it’s convenient. Yes, it helps you publish better packages to npm and spend much less time bundling.

If you were onboard just to answer “what is @pika/pack and how do I use it?” Hopefully you got a good answer, and if you had any questions, feel free to reach out on Twitter. 👋 Thanks for joining!


… Are the tutorial folks gone? OK, now that it’s just us, I can get back to the rant I held off from earlier. This is a longer dive into why I’m excited this exists.

Postscript: the new era of JavaScript

This might have needed to be a separate blogpost, but I figured this new revolution is all heralded by @pika/pack and @pika/web (which I’ll explain later). I just haven’t been this excited about a front-end release in a while, and I’m going to gush a bit. We’ve left the configuration & tutorial behind from here, and we’ll spend some more time discovering why @pika/pack is just the thing the current JS ecosystem needed to take us into the future.

Part 1: npm bundling is hard

Releasing a great npm package even in 2019 is almost an artform (seriously—the fact that the same JavaScript that powers Prettier can run both in Node.js and the browser and your CLI is a stroke of genius). With npm packages there are so many different ways consumers want to use it and so many technical challenges to hurdle that well-bundled npm packages deserve respect.

Because every package author can only focus on one usecase at a time, behind the scenes libraries are all compiled differently and to varying degrees of quality. Some use finely-tuned Babel configs, some use Make & CI (Jenkins) to automate it. Some use Rollup. But even among these configurations, it still takes a commanding knowledge of transpiling and all the different package.json settings to release a great package.

It’s not impossible. It’s just hard.

And even if you do master it, it still takes your umpteenth package until you have it down, by which time you probably won’t have time to go back and maintain all the old methods you’re not using anymore.

Part 2: CommonJS vs ESM

Before I get to @pika/pkg (which, spoiler: it’s great and you should use it), publishing to npm is just the tip of the naval minefield (I’m working on my metaphors; some might come out a little mixed). Beneath the surface, lying in wait, rests the fragmented nature of the JavaScript ecosystem: CommonJS (CJS) vs ES Modules (ESM).

History on this topic warrants another blog post in itself, but in a nutshell, Node.js was built on the CJS method of handling modules (require(), module.exports). Browsers do not speak this language. When Browserify (and later webpack) came along, they changed the game, bringing ES Modules to contemporary JavaScript development (import, export). Then the unthinkable happened: the prometheal fire of ESM lit up browsers (translation: most browsers support ESM now).

…And Node.js followed suit, right?

Unfortunately, despite ESM’s widespread adoption and years of discussion, Node.js still only understands CJS (but ESM may be coming soon… maybe?). The irony of it all is that the bundlers themselves that introduced most of us to ESM are all Node.js-based, so they require everything to be transpiled to CJS just to operate. In other words, the tools we use to make everything possible are also making the switch more difficult.

(Patrick wallet meme) This works in all modern browsers, right? / Yup / And most JS developers have been using ESM for the past 4 years, right / Yup / So we should use ESM for everything! / That makes sense / `import React from 'react'` / `SyntaxError: Unexpected token import``SyntaxError: Unexpected token import` …

To get a sense of how deep the divide is widening: at the time of writing this blog post, about 95% of the 930,826 modules on npm are written in CJS (I subtracted the number of ESM modules according to pika from the total). We can also infer the number of devs that compose using ESM in the significant majority according to theadoption of TypeScript, ES6, and modern JS frameworks. The global number of ESM lines increases every day, as does the number of lines of CJS. The only thing keeping this runaway JS ecosystem train from plummeting off the rails and plowing into the East River is Babel—the tool that transforms basically all of our code, beneath the scenes.

The Amazing Babel-man

Without the folks at Babel and npm making everything better and simpler for all of us, we’d all be lost. Yet despite the leaps and bounds we’ve made, the JS ecosystem is still a messy work in progress, with lots of dirt and loose rebar and the building permits should be coming any day now and some of the old folks complain about the crappy restaurant that used to be here but we can all agree it wasn’t up to health standards and the new restaurant may be a bit pricier but the new food is a bit more ethical and healthier and does what it can to deliver a memorable culinary experience while positively affecting the broken food distribution system (are my metaphors getting better?).

Anyway, theatrics aside, @pika/pack has done something significant: it’s removed a great deal of managing this complexity, allowing us to focus on the simple joy of building while it manages npm and the ecosystem for us.

Part 3: the dawning future of @pika/web

At this point you’re probably thinking “if Browsers all support ES Modules, and we write code in ES Modules, then why do we use bundlers at all?” And your instincts are spot on: we shouldn’t!

“Then why do we?”

The reason bundlers are still used is because now we’re still stuck in a fragmented state. That’s right—React, Angular, and many of the packages you love and depend on are still shipped completely reliant on CJS bundlers. You aren’t able to use ESM for your entire bundle, until 100% of your dependencies are in ESM.

This is a bigger task than it seems at first glance (we’re getting close to 1 million npm packages, remember?). Sensing this need, and this rift, @pika/web aims to be that shiny new future where we ship the same JavaScript we write into the browser, since browsers all support ESM now, slimming bundlers to be nothing more than light cleaner-uppers.

In the @pika/web blog post, they outline all the common questions one might have in a post-webpack (post-Create-React-App, etc.) world, such as “wouldn’t it be slower?” (No, it’s actually faster, at least not on HTTP/2) / “can we still code split?” (The caching is better than you ever dreamed out-of-the-box, and dynamic import() is coming soon).

There’s no reason not to usher in this simpler, lighter, more consistent future. It brings back the halcyon days of just-shove-it-in-a-script-tag but while getting to keep all the npm hotness that’s become indispensible. The only thing in the way of this bright, shiny, incredible future is the massive ecosystem of CommonJS still in use. And the one person for the job, able to tackle this insurmountable task of transforming the entire JS ecosystem is…

@pika/pack

shocked Pikachu meme

If you maintain an npm package please start shipping it with @pika/pack and make the world a better place. ✌️

Stratus Background
StratusUpdate

Sign up for the Stratus Update newsletter

With our monthly newsletter, we’ll keep you up to date with a curated selection of the latest cloud services, projects and best practices.
Click here to read the latest issue.