Edward Thomson

Packaging an Action

May 8, 2020  •  12:04 PM

After having created several actions to use in GitHub Actions workflows, I've settled on a pattern that I really like for packaging them, and in my most recent action, I codified this in a CI process to automate it for me.

The reason that this is important is because GitHub Actions uses a git repository as the distribution mechanism for an action. When you specify an action to run, like:

steps:
- uses: actions/checkout@v2

That indicates that you want to run an action named actions/checkout at version v2. This literally maps to a reference v2 in the repository https://github.com/actions/checkout.

Since actions are actually just Node.js applications, that means that the reference v2 in that repository needs to contain the application, and its dependencies. It needs to actually contain the node_modules directory. Butโ€ฆ you're not supposed to check that in, are you?

No! You're not. At leastโ€ฆ not in your development branch. Now, of course you could technically do this and GitHub Actions would work just fine, but it's going to be messy. Instead, I recommend using a two branch approach:

  1. master is the branch that you work in, just like you would with any application. In this branch, you should not check in your node_modules directory, it should be added to your .gitignore just like any other Node.js application.

  2. dist is the branch that your application is distributed in. This contains the built and packed version of your action, along with any metadata files (your license, README and action.yml, for example).

This keeps the build output separate from your source directory, where it definitely doesn't belong. But it's a little annoying to have to build into a new branch and publish it yourself. And - whenever I see anything that's a manual annoyance, I try to automate it. So I created a GitHub Actions workflow to build my master branch and then publish it into dist.

Here's the simple summary (with comments to explain what's happening):

name: CI

# Run this whenever there's an update to the master branch.
on:
  push:
    branches: [ master ]

jobs:
  #
  # Build (if there's a build step) and run tests to ensure that the
  # new change in master is good.
  #
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Check out source
      uses: actions/checkout@v2

    - name: Build and Test
      run: |
        npm ci
        npm run build --if-present
        npm test
  #
  # Publish the action to the `dist` branch
  #
  publish_action:
    runs-on: ubuntu-latest
    needs: build

    steps:
    - name: Check out source
      uses: actions/checkout@v2

    # Check out the `dist` branch into the `dist` directory.
    - name: Check out distribution branch
      uses: actions/checkout@v2
      with:
        ref: 'dist'
        path: 'dist'

    # Run `npm run pack`, which uses @zeit/ncc to package the action
    # into a single file.  Copy things that we want to publish out of
    # the source directory and into the dist directory (which is where
    # the dist branch is checked out.)
    - name: Package
      run: |
        npm install
        npm run pack
        mkdir -p dist/documentation
        mkdir -p dist/examples
        cp action.yml dist/
        cp README.md dist/
        cp LICENSE.txt dist/
        cp documentation/* dist/documentation/
        cp examples/* dist/examples/

    # Check for changes; this avoids publishing a new change to the
    # dist branch when we made a change to (for example) a unit test.
    # If there were changes made in the publish step above, then this
    # will set the variable `has_changes` to `1` for subsequent steps.
    - name: Check for changes
      id: status
      run: |
        source ../.github/workflows/actions.sh
        if [ -n "$(git status --porcelain)" ]; then
          echo "::set-output name=has_changes::1"
        fi
      working-directory: dist

    # Commit the changes to the dist branch and push the changes up to
    # GitHub.  (Replace the name and email address with your own.)
    # This step only runs if the previous step set `has_changes` to `1`.
    - name: Publish action
      run: |
        git add --verbose .
        git config user.name 'CI User'
        git config user.email 'ci@example.com'
        git commit -m 'Update from CI'
        git push origin dist
      if: steps.status.outputs.has_changes == '1'
      working-directory: dist

This workflow will keep dist updated any time you make changes in the master branch. To reference your action, you can run your/repo@dist in a workflow, or better still, you can create a release off the dist branch. You can even publish your action to the marketplace from those releases.

Before you run this workflow, you'll need to create some commit in the dist branch in your GitHub repository.

You can create an empty one by:

commit_id=$(git commit-tree -m 'Distribution branch' 4b825dc642cb6eb9a060e54bf8d69288fbee4904)
git push origin ${commit_id}:refs/heads/dist

(What's going on here? That's another blog post.)

I think that this workflow is a great way to keep a build of your action up-to-date, but also while keeping it out of your source branch.

libgit2 v1.0 Released

April 1, 2020  •  12:04 PM

After many years of planning, writing code, and fixing bugs, libgit2 v1.0 has been released. ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

libgit2 v1.0

libgit2 is a linkable library for working with git repositories in any application, and you probably interact with the library, even if you don't know it. That's because libgit2 powers your applications and your hosting provider, whether you use GitHub, GitLab, Bitbucket, or nearly anybody else.

The Git community โ€“ really, every software developer โ€“ depends on libgit2. And we've finally released libgit2 v1.0.

If you've followed along with libgit2's development, you might have noticed that we use a fun German word to name every release. (Why German words, you ask? Because most languages don't have a word like "Kummerspeck" โ€“ that's a single word for the concept of over-eating from depression and the resulting weight gain1.)

This release is named "Luftschloss"2, literally "air castle", which refers to an impossible dream that cannot be achieved, like building a castle in the air. This seems appropriate given that we sat down to start planning for libgit2 v1.0 back in 2013.

But thanks to aggressively cutting requirements and some long nights and weekends to really redouble efforts, we've finally shipped it.

In seriousness, we treated v1.0 as a fairly arbitrary milestone, but it is still a milestone. And one of the things that I wanted to look at when we got here was how many people have been involved in making this happen.

Looking through our history, it turns out that there were 427 contributors to the project! Which is incredible! And that's only counting people who've authored a commit - it's not counting the many people who've helped out the project by opening issues, packaging libgit2 for distributions, and using the software in production on a day-to-day basis. ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

Thanks everybody for your hard work and support.

On a side note, when I was taking the list of names and email address pairs in the libgit2 history to get the list of individual contributors, I found some interesting data:

  • Carlos Martรญn Nieto had the most combinations of name and email address signatures, with six. Some of this is the the inevitable ASCII-ization of his name, some of it is due to having many email addresses throughout the many valued years of his contributions.

  • Vicent Martรญ had the next most combinations of name and email addresses, with five. This probably should have only been four, but he accidentally committed something with his user.name set to "The rugged tests are fragile". (Oops!)

  • Sadly, that's not the only embarassing accident of name configuration. I happened to have four different signatures throughout the libgit2 history because I've used three different email addresses throughout the yearsโ€ฆ but also misspelled my own email address once.

It will be fascinating to see how the next ten years of libgit2 development turn out. We have some interesting ideas, like making it easier for GUI clients to use libgit2, increasing feature parity with core git, and leveraging their test cases to ensure that we're fully compatible.

And hopefully I'll spell my name correctly!

  1. Given that I'm averaging about two lunches per day during the pandemic, I think that it's likely that more languages will add a word for this concept.ย 

  2. And given my current lunch habit, we probably would have used "Kummerspeck" for this release, except that it was the name of libgit2 v0.28.ย 

libgit2 v0.99 Released

February 19, 2020  •  12:04 PM

Today we've released libgit2 v0.99. ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

libgit2 v0.99

libgit2 is a linkable library for working with git repositories in any application. It's used by IDEs, GUI clients for Git, hosting providers, scientific computing tools, and even scuba diving log software.

Whether you keep your code on GitHub, GitLab, Bitbucket, or nearly anywhere else, it's libgit2 that does the work every time you click the "merge pull request" button. So if you write software, you've probably used libgit2, even if you don't know it.

And โ€“ finally โ€“ we're getting close to releasing version 1.0.

When I started working on libgit2, eight years ago, I joined a group of developers from all over the world who were working on the project. And shortly after I did, we started planning for all the things that we'd need to do to get to libgit2 version 1.0.

I honestly don't remember a single one of those things.

But I do remember that the biggest thing that we needed to provide to users wasn't features or functionality, it was API consistency.

See, back in ye olde days, we had this idea that we could build the perfect Git library. It would have the perfect API. And once we got there, then we would never need to change it, and that would be our 1.0. And until that happened, we could just keep changing the API until it was perfect. It was, after all, just a prerelease. (Look, it's still version zero-point-something!) And once we got there, all the developers that use libgit2 would be super happy about our great API!

Well, reader, I probably don't have to tell you that people were not, in fact, super happy. It turns out that changing the API on people all the time is not really a recipe for satisfaction.

Hell, even I hated it. I used to be that guy who would bring in new versions of libgit2 into Visual Studio. And then I'd suffer through finding all the places that called git_checkout and needed to be updated whenever we made some API change to "make things better". And then I'd curse myself and my friends who did that.

And as time went on, git became more popular. And more people wanted to build apps that talked to git repositories. And we had more users and more applications building on top of libgit2. And then breaking the API just became cruel.

So after the last release โ€“ v0.28 โ€“ we made a (long overdue) decision: we needed to release a v1.0, finally. And then we'd need to stop breaking the API in minor releases.

So we challenged ourselves to do the refactorings that we wanted to do to improve the API consistency while still keeping the API backward compatible. Sometimes this meant #defines. Sometimes this meant making the old functions proxy to the new functions. And sometimes this just meant realizing that perfection is unattainable. (It got real deep, y'all.)

But we succeeded in our mission: we believe that v0.99 is completely API-compatible with v0.28. This is the first release that we can say that about. Which means that this is the pre-release for v1.0.

So, please, if you're a libgit2 user, give v0.99 a shot. We'll do a period of stabilization to ensure that there are no significant bugs, and then we'll release v1.0 ๐Ÿ”œ.

And once we do that, we'll start looking forward to v2.0, because that's when semver says that we can make breaking API changes again.

Just Kidding!

This is day 31 of my GitHub Actions Advent Calendar. If you want to see the whole list of tips as they're published, see the index.

Advent calendars usually run through Christmas, but I'm going to keep posting about GitHub Actions through the end of December. Consider it bonus content!

Yesterday we built an action that let you send a tweet from a GitHub Actions workflow. Once we built it, you can use it by referencing it in my repository, as ethomson/send-tweet-action@v1. Butโ€ฆ how do people find it?

Well, there's always your favorite search engine โ€“ which, perhaps surprisingly, does a good job of crawling GitHub.1

Search

But we can do better than relying on a search engine. If we publish this action to the GitHub Marketplace, then people who want to build a workflow have a focused place to look for actions that they can use to extend GitHub. The marketplace will even show up in the workflow editor, so that people can find actions easily.

Marketplace Editor

Thankfully, publishing to the marketplace is easy. Once I've created an action and my repository has an action.yml in it, then GitHub will prompt you to create a release, which will let you publish an action to the marketplace.

Publish Prompt

When you create a new release for an action, GitHub will give you the option to publish that release to the marketplace. This will help users find your action with the marketplace search.

Publish

Once I've published this release, it will now exist in the marketplace. Now whenever somebody searches for "send tweet" or "twitter", they'll find the send tweet action.

Marketplace Search

Now people can easily find an action โ€“ whether it's from a search engine, or from the marketplace.


That's the end of my GitHub Actions Advent Calendar! I've had a blast writing it, and I hope that you've learned some things about GitHub Actions along the way. I can't wait to see what you build to automate your GitHub workflows!

  1. This was in private browsing mode, and on a different wireless network than the one that I was working on yesterday when I was searching for Twitter APIs and GitHub Actions documentation. This hardly suggests that this will be the general results, though.ย 

This is day 30 of my GitHub Actions Advent Calendar. If you want to see the whole list of tips as they're published, see the index.

Advent calendars usually run through Christmas, but I'm going to keep posting about GitHub Actions through the end of December. Consider it bonus content!

Yesterday we built an action that posted to a GitHub issue. It showed how easy it is to work with GitHub APIs from within an action. But actions are generally written in JavaScript, so they can easily integrate with APIs from other services. This lets you create actions that can create an issue in Jira, send a text message, or upload security information about your project.

One of the things that I want to automate is sending a tweet โ€“ this is something I do whenever I release a new version of something, to announce its availability.

And I'd love an action that I can use in a workflow to automate this. If you look in the GitHub Actions Marketplace, you can see that there is actually an action already that can tweet. Unfortunately, it's built as a docker container.

You can create actions as either a Docker container or a JavaScript application. It might seem easier to get started with a container-based action, but there are limitations: only Linux based workflows (those that run on ubuntu-latest) can use them. Using a JavaScript action ensures that your action can be used in any workflow.

So if I want to send a tweet whenever any of my projects are released โ€“ including the ones that run on macOS or Windows โ€“ then I'll need to create a JavaScript action that can send a tweet.

To do this, I'll start with the sample TypeScript action and add in the twitter npm package so that I can communicate with the Twitter API. This is quite straightforward โ€“ I'll need to take the authentication as inputs to the action and use those to create a Twitter API instance.

const twitter = new Twitter({
  consumer_key: core.getInput('consumer-key'),
  consumer_secret: core.getInput('consumer-secret'),
  access_token_key: core.getInput('access-token'),
  access_token_secret: core.getInput('access-token-secret')
})

Then I can send a tweet by calling the /statuses/update API. I'll take another input to the action that will be the status to tweet.

twitter.post(
  '/statuses/update',
  {status: core.getInput('status')},
  (error, data, response) => { if (error) throw error }
)

Once I've got that written, all I need to do is package it up โ€“ creating a distribution branch, pushing that up to GitHub, and then creating a release.

You can find the finished product at github.com/ethomson/send-tweet-action, so you can use ethomson/send-tweet-action@v1 as part of a workflow.

Now any time I push to this repository, it will send a tweet.

You can use this action to send a tweet whenever a release is created, a wiki is updated, or a deployment occurs. Or, just for fun, like what I've done.

It's easy to integrate other APIs into an action to let you automate other tools based on activities in your repository. I can't wait to see what you build!