Skip to main content

Testing

Testing​

As of v1.19.2, GitProxy uses Mocha (ts-mocha) as the test runner, and Chai for unit test assertions. End-to-end (E2E) tests are written in Cypress, and some fuzz testing is done with fast-check.

Unit testing with Mocha and Chai​

Here's an example unit test that uses Chai for testing (test/testAuthMethods.test.js):

// Import all the test dependencies we need
const chai = require('chai');
const sinon = require('sinon');
const proxyquire = require('proxyquire');

// Import module that contains the function we want to test
const config = require('../src/config'); 

// Allows using chain-based expect calls 
chai.should();
const expect = chai.expect;

describe('auth methods', async () => {
  it('should return a local auth method by default', async function () {
    const authMethods = config.getAuthMethods();
    expect(authMethods).to.have.lengthOf(1);
    expect(authMethods[0].type).to.equal('local');
  });

  it('should return an error if no auth methods are enabled', async function () {
    const newConfig = JSON.stringify({
      authentication: [
        { type: 'local', enabled: false },
        { type: 'ActiveDirectory', enabled: false },
        { type: 'openidconnect', enabled: false },
      ],
    });

    const fsStub = {
      existsSync: sinon.stub().returns(true),
      readFileSync: sinon.stub().returns(newConfig),
    };

    const config = proxyquire('../src/config', {
      fs: fsStub,
    });

    // Initialize the user config after proxyquiring to load the stubbed config
    config.initUserConfig();

    expect(() => config.getAuthMethods()).to.throw(Error, 'No authentication method enabled');
  });

  it('should return an array of enabled auth methods when overridden', async function () {
    const newConfig = JSON.stringify({
      authentication: [
        { type: 'local', enabled: true },
        { type: 'ActiveDirectory', enabled: true },
        { type: 'openidconnect', enabled: true },
      ],
    });

    const fsStub = {
      existsSync: sinon.stub().returns(true),
      readFileSync: sinon.stub().returns(newConfig),
    };

    const config = proxyquire('../src/config', {
      fs: fsStub,
    });

    // Initialize the user config after proxyquiring to load the stubbed config
    config.initUserConfig();

    const authMethods = config.getAuthMethods();
    expect(authMethods).to.have.lengthOf(3);
    expect(authMethods[0].type).to.equal('local');
    expect(authMethods[1].type).to.equal('ActiveDirectory');
    expect(authMethods[2].type).to.equal('openidconnect');
  });
});

Core concepts to keep in mind when unit testing JS/TS modules with Chai:

Stub internal methods to make tests predictable​

Functions often make use of internal libraries such as fs for reading files and performing operations that are dependent on the overall state of the app (or database/filesystem). Since we're only testing that the given function behaves the way we want, we stub these libraries.

For example, here we stub the fs library so that "reading" the proxy.config.json file returns our mock config file:

// Define the mock config file
const newConfig = JSON.stringify({
  authentication: [
    { type: 'local', enabled: true },
    { type: 'ActiveDirectory', enabled: true },
    { type: 'openidconnect', enabled: true },
  ],
});

// Create the stub for `fs.existsSync` and `fs.readFileSync`
const fsStub = {
  existsSync: sinon.stub().returns(true),
  readFileSync: sinon.stub().returns(newConfig),
};

This stub will make all calls to fs.existsSync to return true and all calls to readFileSync to return the newConfig mock file.

Then, we use proxyquire to plug in the stub to the library that we're testing:

const config = proxyquire('../src/config', {
  fs: fsStub,
});

// Initialize the user config after proxyquiring to load the stubbed config
config.initUserConfig();

Finally, when calling the function we're trying to test, the internal calls will automatically resolve to the values we chose.

Setup and cleanup​

before and beforeEach, after and afterEach are testing constructs that allow executing code before and after each test. This allows setting up stubs before each test, making API calls, setting up the database - or otherwise cleaning up the database after test execution.

This is an example from another test file (test/addRepoTest.test.js):

before(async function () {
  app = await service.start();

  await db.deleteRepo('test-repo');
  await db.deleteUser('u1');
  await db.deleteUser('u2');
  await db.createUser('u1', 'abc', 'test@test.com', 'test', true);
  await db.createUser('u2', 'abc', 'test2@test.com', 'test', true);
});

// Tests go here

after(async function () {
  await service.httpServer.close();

  await db.deleteRepo('test-repo');
  await db.deleteUser('u1');
  await db.deleteUser('u2');
});

Focus on expected behaviour​

Mocha and Chai make it easy to write tests in plain English. It's a good idea to write the expected behaviour in plain English and then prove it by writing the test:

describe('auth methods', async () => {
  it('should return a local auth method by default', async function () {
    // Test goes here
  });

  it('should return an error if no auth methods are enabled', async function () {
    // Test goes here
  });

  it('should return an array of enabled auth methods when overridden', async function () {
    // Test goes here
  });
});

Assertions can also be done similarly to plain English:

expect(authMethods).to.have.lengthOf(3);
expect(authMethods[0].type).to.equal('local');

Unit testing coverage requirement​

All new lines of code introduced in a PR, must have over 80% coverage (patch coverage). This is enforced by our CI, and generally a PR will not be merged unless this coverage requirement is met. Please make sure to write thorough unit tests to increase GitProxy's code quality!

If test coverage is still insufficient after writing your tests, check out the CodeCov report after making the PR and take a look at which lines are missing coverage.

E2E testing with Cypress​

Although coverage is currently low, we have introduced Cypress testing to make sure that end-to-end flows are working as expected with every added feature.

This is a sample test from cypress/e2e/repo.cy.js:

describe('Repo', () => {
  beforeEach(() => {
    // Custom login command
    cy.login('admin', 'admin');

    cy.visit('/dashboard/repo');

    // prevent failures on 404 request and uncaught promises
    cy.on('uncaught:exception', () => false);
  });

  describe('Code button for repo row', () => {
    it('Opens tooltip with correct content and can copy', () => {
      const cloneURL = 'http://localhost:8000/finos/git-proxy.git';
      const tooltipQuery = 'div[role="tooltip"]';

      cy
        // tooltip isn't open to start with
        .get(tooltipQuery)
        .should('not.exist');

      cy
        // find the entry for finos/git-proxy
        .get('a[href="/dashboard/repo/git-proxy"]')
        // take it's parent row
        .closest('tr')
        // find the nearby span containing Code we can click to open the tooltip
        .find('span')
        .contains('Code')
        .should('exist')
        .click();

      cy
        // find the newly opened tooltip
        .get(tooltipQuery)
        .should('exist')
        .find('span')
        // check it contains the url we expect
        .contains(cloneURL)
        .should('exist')
        .parent()
        // find the adjacent span that contains the svg
        .find('span')
        .next()
        // check it has the copy icon first and click it
        .get('svg.octicon-copy')
        .should('exist')
        .click()
        // check the icon has changed to the check icon
        .get('svg.octicon-copy')
        .should('not.exist')
        .get('svg.octicon-check')
        .should('exist');

      // failed to successfully check the clipboard
    });
  });
});

Here, we use a similar syntax to Mocha to describe the behaviour that we expect. The difference, is that Cypress expects us to write actual commands for executing actions in the app. Some commands used very often include visit (navigates to a certain page), get (gets a certain page element to check its properties), contains (checks if an element has a certain string value in it), should (similar to expect in unit tests).

Custom commands​

Cypress allows defining custom commands to reuse and simplify code.

In the above example, cy.login('admin', 'admin') is actually a custom command defined in /cypress/support/commands.js. It allows logging a user into the app, which is a requirement for many E2E flows:

Cypress.Commands.add('login', (username, password) => {
  cy.session([username, password], () => {
    cy.visit('/login');
    cy.intercept('GET', '**/api/auth/me').as('getUser');

    cy.get('[data-test=username]').type(username);
    cy.get('[data-test=password]').type(password);
    cy.get('[data-test=login]').click();

    cy.wait('@getUser');
    cy.url().should('include', '/dashboard/repo');
  });
});

Fuzz testing with fast-check​

Fuzz testing helps find edge case bugs by generating random inputs for test data. This is very helpful since regular tests often have naive assumptions of users always inputting "expected" data.

Fuzz testing with fast-check is very easy: it integrates seamlessly with Mocha and it doesn't require any additional libraries beyond fast-check itself.

Here's an example of a fuzz test section for a test file (testCheckRepoInAuthList.test.js):

const fc = require('fast-check');

// Unit tests go here

describe('fuzzing', () => {
  it('should not crash on random repo names', async () => {
    await fc.assert(
      fc.asyncProperty(
        fc.string(),
        async (repoName) => {
          const action = new actions.Action('123', 'type', 'get', 1234, repoName);
          const result = await processor.exec(null, action, authList);
          expect(result.error).to.be.true;
        }
      ),
      { numRuns: 100 }
    );
  });
});

Writing fuzz tests is a bit different from regular unit tests, although we do still assert whether a certain value is correct or not. In this example, fc.string() indicates that a random string value is being generated for the repoName variable. This repoName is then inserted in the action to see if the processor.exec() function is capable of handling these or not.

In this case, we expect that the result.error value is always true. This means that the exec flow always errors out, but never crashes the app entirely. You may also want to test that the app is always able to complete a flow without an error.

Finally, we have the numRuns property for fc.assert. This allows us to run the fuzz test multiple times with a new randomized value each time. This is important since the test may randomly fail or pass depending on the input.