How to Setup a JQuery Project as a Node Module
Sometimes it makes sense to turn a JavaScript project into a module which can be imported by other projects. There are a number of pages which describe how to link a project locally using npm or publish the project to a package index such as npm. However, a project which will become a dependency for other projects must also setup processes such as testing, coverage and documentation or it risks causing more problems than it solves. Testing, code coverage and documentation are also best practices for open-source collaboration and publication.
Plan of Attack
This article will describe how to setup a node module for a jquery based project with all the tools, services and configuration files necessary for long-term maintenance in 15 steps. For your convenience, we have also put together a python-based command line tool which will automatically generates all the necessary files and allow you to skip steps 2–11 (described below). For the purposes of this guide, we will be using Karma, Webpack, Babel, Mocha, Chai, Sinon, PhantomJS, NYC (previously Istanbul), Coveralls, StandardJS and Docsify, but each of these tools have alternatives if you prefer a different testing setup.
Background
One of the major issues with JavaScript development as of September 2018 is the fact that the standard for JavaScript has outpaced the improvement of the core software used to run JavaScript in the wild. Although ES6 has been the standard since 2015, neither browsers (running front-end scripts) nor nodejs (running back-end scripts) accept the syntax of ES6. So, everything developed with the latest best practices must be compiled by a program that translates code into an older acceptable format first. On top of this, browsers, nodejs and all the tools used to test software each have their own mutually incompatible syntax, so you need different transpilers for different parts of the process. This creates a DevOps configuration smorgsborg and often numerous days of frustrating google searches over stackoverflow before any real code is written.
Prerequisites
This article will cover installation and setup of all the tools listed above along with the steps necessary to link or publish a project. However, it assumes that you have already installed nodejs, nodes package manager (npm), git (or mercurial) and you have a github (or bitbucket) account. If you have not already installed these prerequisites, it is recommended to install nvm before installing nodejs and npm, as nvm will allow you to have different node environments should you need it and easily delete them if one gets screwed up.
Step 1: Install NPM Modules
We will be installing the tools we need as globally accessible modules. This means that we will be able to access these tools on other projects without having to reinstall these tools. Although most other tutorials instruct you to use a local installation process, such a process results in every project folder being 200MB+ prior to even writing a line of code. By installing globally, that number is reduced to a more reasonable ~10MB since there are only a few tools which must be installed locally or they will not function properly. However, if you aren’t running your development off a meager Chromebook using GalliumOS and prefer NPM bloat, you can skip this step and we have included all the global dependencies in the package.json file in the next step.
npm install -g chai coveralls docsify-cli karma karma-chai \
karma-cli karma-coverage karma-mocha karma-mocha-reporter \
karma-phantomjs-launcher karma-sinon karma-webpack \
mocha nyc phantomjs-prebuilt sinon standard webpack webpack-cliPocketLab (or How to Skip Steps 2–11)
To make this process easier and reduce the possibility of error in transcription, we have added an npm module builder to our command-line tool called pocketlab. PocketLab provides popup frameworks for other types of projects and python modules as well as making it easier to deploy code to Heroku and AWS. But, for the purpose of this guide only one command is needed after installing pocketlab.
pip install pocketlab
lab init <module-name> - jqueryOnce the lab script finishes, you will then need to run the following npm commands to install the remaining dependencies which must be installed locally. After that you can go to step 12:
npm install @babel/core @babel/preset-env babel-loader
npm install - only=prodWarning: PocketLab currently requires prior installation of python 3+.
Step 2: Create package.json
Node based javascript simply doesn’t work without the configuration file package.json and every module has package.json to fully describe it to other modules. Here is the text for the package.json which you need to create in the root of your project to cover all the tools we are using. If you have a preferred blueprint for your javascript development, then you can use it instead, but you will need to add all the scripts and devDependencies listed here in order to follow along with this guide. Also, please note that there is a lot of placeholder text which you will need to fill in to match your project details such as <module-name>, <user-name>, <repo-name> and user@domain.com. Also, due to Medium’s auto-formatting, you will need to run a replace text to remove all the spaces after the character ^. Once you have created your package.json file and replaced the placeholder text, run ```npm install```.
{
"name": "<module-name>",
"version": "0.0.1",
"description": "",
"author": "<user-name> <user@domain.com>",
"contributors": [
{
"name": "<user-name> <user@domain.com>"
}
],
"bugs": {
"url": "https://github.com/<user-name>/<repo-name>/issues"
},
"repository": {
"type": "git",
"url": "https://github.com/<user-name>/<repo-name>"
},
"homepage": "https://github.com/<user-name>/<repo-name>",
"dependencies": {
"jquery": "^3.3.1"
},
"license": "MIT",
"main": "dist/<module-name>.min.js",
"scripts": {
"test": "karma start karma.config.js",
"build": "webpack --config webpack.config.js",
"prepare": "npm run build",
"coverage": "nyc report --reporter=text-lcov | coveralls"
},
"devDependencies": {
"@babel/core": "^7.0.1",
"@babel/preset-env": "^7.0.0",
"babel-loader": "^8.0.2",
"chai": "^4.1.2",
"coveralls": "^3.0.2",
"docsify-cli": "^4.2.1",
"karma": "^3.0.0",
"karma-chai": "^0.1.0",
"karma-cli": "^1.0.1",
"karma-coverage": "^1.1.2",
"karma-mocha": "^1.3.0",
"karma-mocha-reporter": "^2.2.5",
"karma-phantomjs-launcher": "^1.0.4",
"karma-sinon": "^1.0.5",
"karma-webpack": "^3.0.4",
"mocha": "^5.2.0",
"nyc": "^13.0.1",
"phantomjs-prebuilt": "^2.1.16",
"sinon": "^6.2.0",
"standard": "^11.0.1",
"webpack": "^4.17.2",
"webpack-cli": "^3.1.0"
}
}
Step 3: Create karma.config.js
Karma is a tool which creates a session with a browser installed on your device so that you can test code which requires the “window” and “document” variables such as JQuery. There are ways to create these variables within node using a module called jsdom, but to my knowledge there is no way to properly import jquery to bind to these variables without changing your source code to do so. Which means it is not a viable tool for testing code that is intended to be run in the browser. As a result, we are using Karma and the headless browser PhantomJS to generate real browser environments to test code. Another perfectly viable alternative is to use Chrome by installing karma-chrome-launcher instead… but it takes chrome longer to launch and it can be quite annoying for the testing program to popup and quickly close a chrome window. In order to for karma to work properly, it must be configured properly. Here is the text for the karma.config.js file you should create in the root of the project which will interact correctly with our other tools.
module.exports = function(config) {
config.set({
browsers: ['PhantomJS'],
files: [
'./test/*.js'
],
frameworks: ['mocha', 'chai', 'sinon'],
preprocessors: {
'./test/*.js': ['webpack']
},
reporters: ['mocha', 'coverage'],
webpack: {
module: {
rules: [
{ test: /\.js/, exclude: /node_modules/, loader: 'babel-loader' }
]
},
watch: true,
mode: 'none'
},
webpackServer: {
noInfo: true
},
singleRun: true,
coverageReporter: {
includeAllSources: true,
dir: 'coverage/',
reporters: [
{ type: "html", subdir: "html" },
{ type: 'text-summary' }
]
}
});
};
Step 4: Create webpack.config.js
Webpack is a tool which compiles node-based javascript into javascript which will run in the browser. Along with the help of Babel, this means that you can use any other node module as a dependency for your browser scripts and it will only include the code which is required. With webpack, you do not need to add lists of minified files to the body of your html and carefully order them to account for their dependencies. Nor do you need to add all your front-code you write over years of different projects just to have the parts of it that need for your current project. Here is the text for the webpack.config.js you need to create in the root of the project configured for our project. Please notice that jquery has been added as an external file. This means that you will need to include jquery.min.js before your bundled file or you will get an error message in your console log that says $ is undefined. You can remove jquery from externals if you wish, but then webpack will just recompile jquery.min.js and add it to the beginning of your final file.
const path = require('path');module.exports = {
entry: './src/<module-name>.js',
output: {
path: path.resolve(__dirname, 'dist'),
filename: '<module-name>.min.js',
library: '<module-name>' // to enable as object in window
},
mode: 'production',
externals: {
jquery: 'jQuery'
// add external dependencies here
},
module: {
rules: [
{
test: /\.js$/,
exclude: '/node_modules/',
use: {
loader: 'babel-loader',
options: {
presets: ['@babel/preset-env']
}
}
}
]
}
};
Step 5: Create .babelrc
Babel is a tool which translates javascript from one dialect (ES6 in our case) into another and may be the most popular module on npm for that reason. Babel is used by just about all the other tools in this guide and it is also the only tool family which does not get installed globally. While trying to get these tools to work, all sorts of errors seem to occur if babel is not installed locally. So, it is included in the package.json file dependencies for development. The configuration for Babel can be complex, but for our environment, we will use the standard preset environment. Here is the text for the .babelrc file you need to create in the root of the project.
{
"presets": [ "@babel/preset-env" ]
}
Step 6: Create .gitignore
To prevent all the installation files from bloating the commit records in the .git folder, being pushed to the remote repo and interfering with any other collaborators own installations, we need to use .gitignore generously. Gitignore is also vital to prevent credentials like the ones we get from coveralls.io in step 14 from becoming public. Here is the text for .gitignore file you need to create in the root of your project which also excludes files generated by the very nice IDEs made by Jetbrains and other useful data storage services. Feel free to add more excluded patterns to your .gitignore. If you opt to use mercurial instead as your VCS, then this file is called .hgignore and the patterns are slightly different. Most notably, mercurial uses regular expression, so any use of . in a file name must first be escaped by the character \ such as \.coveralls\.yml or it will be misinterpreted.
################# .gitignore ################################## data files #################
data/################# distro files #################
site/################# unit testing #################
.nyc_output/
.coveralls.*
.coverage
.coverage.*
.cache
coverage.xml################# dependencies #################
node_modules/
package-lock.json################# dev files ################################# version control ################################# IDE files #################
.DS_STORE
.netbeans
nbproject
.idea
.node_history
venv/
venv3/################# backup sync #################
desktop.ini
*.gdoc
.dropbox
Thumbs.db
Step 7: Create .npmignore
If you plan to publish your module, you will want to use .npmignore to exclude all the same files you exclude with .gitignore as well as some not covered by .gitignore. There is no reason that your coverage and testing files need to be included in the package other install. People who wish to collaborate on your project will instead clone (or preferably fork) those files from the repository. If you plan to only use the module locally, then you can skip this file as it does not seem to have any effect on npm link. Here is the text for the .npmignore file you can create in the root of the project.
################# .npmignore ################################## data files #################
data/################# distro files #################
site/################# unit testing #################
coverage/
.nyc_output/
.coveralls.*
.coverage
.coverage.*
.cache
coverage.xml################# dependencies #################
*.swp
npm-debug.log
node_modules/
package-lock.json################# dev files #################
test/
docs/
.babelrc
webpack.config.js
karma.config.js################ version control ################
.gitignore
.git/################# IDE files #################
.DS_STORE
.netbeans
nbproject
.idea
.node_history
venv/
venv3/################# backup sync #################
desktop.ini
*.gdoc
.dropbox
Thumbs.db
Step 8: Create test/<module-name>-spec.js
Unittests are key to long-term code maintenance and collaboration. Unittests are also required to determine code coverage percentages using the code coverage tools we have installed. For unittesting, we will use the mocha module since it generates nice looking reports along with the chai and sinon modules for convenient unittesting syntax. Although these are some of the most popular modules, other tools like jasmine can also be used. In which case, all references to mocha need to be replaced with jasmine and additional jasmine connectors need to be installed. By convention, unittests take the name of the file being tested and add the suffix -spec.js. So we will create an initial unittest file for the project in a sub-folder called ‘test’ which runs a single test that cannot fail. Here is the text for test/<module-name>-spec.js you need to create.
import * as <module-name> from '../src/<module-name>.js'// https://www.chaijs.com/guide/stylesdescribe('<module-name>', function() {
describe('Placeholder()', function() {
it('should do nothing yet', function() {
expect(2).to.equal(2)
});
});
});
Step 9: Create src/<module-name>.js
To run tests and build a distribution file, you will need to have some source code to add to the project or create some placeholder code now. For the purposes of this guide, we will add the source code to a file named after the module in a folder called src so that it matches the settings in our configuration files. Either copy your code or add the following text which you can update later to the file path src/<module-name>.js
/*!
* @name <module-name>
* @description A Brilliant Javascript Module For The Browser
* @author <user-name>
* @contributors <user-name> <user@domain.com>
* @license MIT // MIT, BSD, ALv2, GPLv3+, LGPLv3+, SEE LICENSE IN LICENSE.txt
* @version 0.0.1
* @copyright 2018 <user-name>
* @email user@domain.com
* @url https://github.com/<user-name>/<repo-name>
* @preserve
*/
// import dependencies
import * as $ from 'jquery'// placeholder class
export default class Placeholder {
constructor() {
// other static properties
}
log(msg) {
console.log(this.name + ': ' + msg)
}
get name() {
return $('#name').text()
}
set name(name) {
$('#name').text(name)
}
}
Step 10: Create .coveralls.yml
To report the outcome of our unittests to the code coverage service we will use, you need to add a configuration file to the root of your project with an access token you will get from coveralls.io (in Step 15). Before you get the token specific for your repository, you can setup a placeholder file. Here is the text for .coveralls.yml you can use as a placeholder.
repo_token: abcdefGHIJKLmnopqrSTUVWXyz0123456789Step 11: Create README.md
To document your project, instruct others on how to install it, use it and collaborate with you, it is important to have a number of items covered in your project’s README.md file. This file will become not only the default first page in GitHub, but it will also act as the description of the project on its npm package page if you publish the module. Here is the text for a sample README.md template that you can create in the root of your project which includes some of the convenient tables and banners that are commonly used to show off your project.
# <module-name>
_A Brilliant Javascript Module For The Browser_<table>
<tbody>
<tr>
<td><b>Downloads</b></td>
<td><a href="https://www.npmjs.com/package/<module-name>">https://www.npmjs.com/package/<module-name></a></td>
</tr>
<tr>
<td><b>Source</b></td>
<td><a href="https://github.com/<user-name>/<repo-name>">https://github.com/<user-name>/<repo-name></a></td>
</tr>
<tr>
<td><b>Documentation</b></td>
<td><a href="https://<user-name>.github.io/<repo-name>">https://<user-name>.github.io/<repo-name>/</a></td>
</tr>
</tbody>
</table>## Intro
Why you should use this brilliant tool## Installation
From NPM:
```shell
$ npm install <module-name> --only=prod
```
From GitHub:
```shell
$ git clone https://github.com/<user-name>/<repo-name>
$ cd placeholder
$ npm install --only=prod
```## Usage
How to import and use each class or function in the module...## Dev Installation
```shell
$ npm install --only=dev
# or
$ npm install -g chai coveralls docsify-cli karma karma-chai karma-cli karma-coverage karma-mocha karma-mocha-reporter karma-phantomjs-launcher karma-sinon karma-webpack mocha nyc phantomjs-prebuilt sinon standard webpack webpack-cli
$ npm install @babel/core @babel/preset-env babel-loader
```## Test
```shell
$ npm test
```## Build
```shell
$ npm run build
```## Report
```shell
$ npm run coverage
```## Collaboration Notes
A collaborator should always **FORK** the repo from the main master and fetch changes from the upstream repo before making pull requests. Please add unittests and documentation for any features added in the pull request.
Root Folder
At this point, your root folder should look something like this:
node_modules/
test/
.babelrc
.coveralls.yml
.gitignore
.npmignore
karma.config.js
package.json
package-lock.json
README.md
webpack.config.jsOr, if you skipped steps 2–11 and used the pocketlab tool, you will see the following folders and files in your root folder. pocketlab also includes a number of other helpful folders and files for module development:
docs/
node_modules/
src/
test/
.babelrc
.coveralls.yml
.gitignore
.hgignore
.npmignore
CHANGELOG.md
karma.config.js
LICENSE.txt
package.json
package-lock.json
README.md
webpack.config.js
Step 12: Initialize Git
To store a remote copy of your project we will be using GitHub, but it is perfectly acceptable to use Bitbucket instead. The advantage of Bitbucket is that it allows unlimited private repositories, so it is quite convenient even if not as famous. You will need to create a repository for your project, ideally with the same name as the module-name before you can push to it. And in step 14, you will link this repository to your coveralls.io account so that it can monitor your code coverage. Once you have setup the remote repository, you will initialize your local git repository, commit it and push the records to the remote with the following commands:
git init
git remote add origin https://github.com/<user-name>/<repo-name>.git
git add -A
git commit -m 'initial commit;'
git push origin masterIf you opt to use mercurial with a Bitbucket account, you will instead need to create an .hgrc file inside the .hg folder mercurial uses to track records and add the following configuration text to it in order to push the local version to the remote repository:
# repository config (see 'hg help config' for more info)
[paths]
default = https://github.com/<user-name>/<repo-name>
[ui]
# name and email (local to this repository, optional), e.g.
# username = Jane Doe <jdoe@example.com>
Step 13: Test & Build
Once some sample source code is in place, it is now possible to run the test and build scripts in your package.json file. The test script will utilize karma, babel, webpack and phantomjs to compile the code and create an environment in a browser to run the unittests in the test/ folder. Mocha, chai, sinon and nyc will produce a report for each test and total coverage which will look pretty boring with our placeholder code. But boring is good if it works and we can then get to writing actual code and real unittests. Testing and building is done with the following commands and will produce files with the coverage report in the coverage/ folder.
npm test
npm run build
And every so often it is useful to run the standard module before or after testing. StandardJS is a module which runs through your code and creates an itemized list of all the elements of your code which do not conform to Javascript Standard Style. Depending on how you code, this list may be long, so standard can be run with the — fix flag to attempt to automatically correct your syntax.
standard /src
// standard /src --fixStep 14: Setup Coverage
Coveralls is a service that visualizes and tracks code coverage. It also integrates with shields.io to report code coverage to your project repo and npm package page. But, it requires sharing quite a bit of access to your github account details and commit records. So, if publication of your module is not relevant, then you can skip this step. Otherwise, go onto coveralls.io and setup an account using github oauth login. Once inside you will need to navigate to the add repos page and select one of your repositories to enable. If your repository is part of an organization, then you will need to first hit the enable private repos button on one of the lower bottom bars, provide additional github oauth permissions and then the refresh repos button to see your organization repos. Once you enable the repo on github for your module, coveralls will provide you with a root_token which you need to copy to the .coveralls.yml file we setup earlier. Once that is done, you can run the coverage script and the report from your previous test which was generated by nyc will appear in coveralls and the outcome percentage on the badges on your README.md file.
npm run coverageStep 15: Link & Publish
Finally, your module is ready to be used by other projects and/or published to the npm index. To link your project, simply run ```npm link``` in the root of the project. This will automatically trigger the prepare script in the package.json which means everything will be rebuilt. If you wish to import this project into your other projects, then you can run ```npm link <module-name>``` to install a symbolic link to this project and access the code. To publish your project, you will need to setup an npm account and retrieve a key from npm to access your account using ```npm adduser```. Once you have connected your local device to npm, you can publish your code.
npm publish --dry-run // to check files built for public release
npm publish
Step 16: Documentation on GitHub.io [Extra Credit]
Docsify is a documentation tool which allows you to create a variety of webpages beyond the single README.md file and host the entire collection of pages on GitHub’s sister documentation site GitHub.io. Although it works for even just a single page (and looks much better than GitHub), this tool especially comes in handy when there are many different classes or functions in your module and you need a place to explain all the methods, options and arguments. Plus, as the owner of a GitHub page, you automatically get to upload data for that page to its corresponding GitHub.io url. If you initialized your project with pocketlab, then your initial README.md file has already been placed in the docs/ folder and you are ready to publish your initial documentation. Otherwise, you will need to initialize docsify with ```docsify init ./docs``` and copy your README.md file to the docs/ folder after initialization. Once you are happy with the wording of the README.md file in docs/, then you need to commit and push your code to GitHub and follow these instructions to enable the documentation.
Wrap Up
That’s it! You should now have a project folder for browser based JavaScript that’s setup to test, build and collaborate with others using the best practices of ES6 and code development. Some people also use a program like Travis for continuous integration and/or setup daemons locally to watch for changes and automatically rebuild, but we will leave those instructions for another guide.
Feedback
This guide has been composed from two recent module projects in September 2018. It is possible a step is missing or a configuration is incorrect. It is also possible some of the tools are no longer active. So, if you spot errors or encounter problems or have suggestions (for replacements to discontinued tools), please feel free to comment.
