Managing Gmail Configuration for Labels and Filters as Code

This post is based on my previous thread, now further expanded with additional thoughts.

why not using Gmail UI to manage

I feel like Gmail hasn’t improved the searching/filtering/labeling part in years. It’s a poor experience in general, especially for advanced usages.

confusing query builder

For example, to filter emails by compound conditions, I must learn a special syntax and put the query in the input labeled “Has the words”.

gmail search experiences

In comparison, here is a common general-purpose query builder component (react-querybuilder). It covers all types of conditions here, providing accurate information and precise control. Why can’t we have something like this, at least before inventing a query language?

react-querybuilder

bad importing experiences

Importing filter rules is just a disastrous experience.

• The embarrassing design feels like it’s from the 90s: users have to click 3 times for “import rules”, “choose file”, and “open file”.
• The modal auto closes itself after finishing, no matter succeeded or failed.
• In many cases, the operation fails with no reason provided, hangs forever, or even crashes the whole tab.

import failing with no reason provided

nested labels not really nested

Another surprise is that nested labels (e.g. news/a, and news/b) do not actually inherit. Therefore, you can’t easily filter all mail with labels news/a or news/b. The only way is to add a parent label news to all children mails as well.

config as code

gmail-britta and gmailctl both attempt to manage Gmail config as code. gmail-britta is the pioneer, but unfortunately has been unmaintained, while gmailctl is actively maintained.

gmailctl uses jsonnet, which is a templating language that compiles json-compatible data. With a templating language, it’s possible to define helper functions, reuse common filter actions, dynamically generate labels/filters, and split the configurations into different files.

sample decent configuration starter set

// subscriptions.libsonnet

local subscriptions = [
  { sender: "Ruby Weekly" },
  { sender: "LLVM Weekly" },
  // ...
];

local rules = [{
  filter: { from: s.sender },
  actions: {
    markSpam: false,
    markImportant: false,
    category: "updates",
    labels: [ "subscriptions", "subscriptions/" + s.sender ]
  },
} for s in subscriptions];

// enforce the parent label
local labels = [{ name: "subscriptions" }] + [{ name: "subscriptions/" + s.sender } for s in subscriptions];

{
  labels: labels,
  rules: rules,
}

// invoices.libsonnet

local action = {
  markSpam: false,
  markImportant: false,
  category: "updates",
  labels: [
    "invoices"
  ]
};

local rules = {
  {
    filter: {
      or: [
        { from: "Google Payments" },
        { from: "@intl.paypal.com" },
        // ...
      ]
    },
    actions: action,
  },
  {
    filter: {
      and: [
        { from: "Apple" },
        { subject: "receipt" }
      ]
    },
    actions: action,
  },
  {
    filter: {
      and: [
        { from: "Microsoft Azure" },
        { subject: "billing" }
      ]
    },
    actions: action,
  },
}

local labels = [{
  name: "invoices",
  color: {
  background: "#ffad46",
  text: "#ffffff"
  }
}]

{
  labels: labels,
  rules: rules,
}

// config.jsonnet

local lib = import 'gmailctl.libsonnet';

local me = 'my@self.com';

local subscriptions = import 'subscriptions.libsonnet';
local subscriptionLabels = subscriptions["labels"];
local subscriptionRules = subscriptions["rules"];

local invoices = import 'invoices.libsonnet';
local invoiceLabels = invoices["labels"];
local invoiceRules = invoices["rules"];

{
  version: "v1alpha3",
  labels: [
    { name: "Notes" }, // gmail native ones
  ] + subscriptionLabels + invoiceLabels,
  rules: [
    // directly TO me
    {
      filter: lib.directlyTo(me), // a helper to match me only in TO, not in CC/BCC
      actions: {
        markImportant: true
        labels: ["p0", "p0/directed"]
      },
    },
    // CC/BCC me
    {
      filter: {
        or: {
          { cc: me },
          { bcc: me },
        }
      },
      actions: {
        markImportant: true
        labels: ["p1", "p1/involved"]
      },
    },
  ] + subscriptionRules + invoiceRules
}

the migration flow

With this flow, you can migrate your current UI-based Gmail configuration to a declarative one.

1. prepare the configurations
   1. install and setup gmailctl.
   2. download and back up your current config with gmailctl download > ~/.gmailctl/config.jsonnet.
   3. read the docs and compose your own configurations.
   4. verify your change with gmailctl diff
2. a fresh start
   1. (optional) if you want to manage labels with gmailctl config as well
      1. (optional, and with caution) remove all labels from all mails in Gmail setting.
      2. use gmailctl apply to apply labels. This command also create filters but we can batch-delete them later anyway.
   2. re-apply all filters:
      1. navigate to Gmail settings/Filter and Blocked Addresses.
      2. (with caution) remove all existing filters.
      3. use gmailctl export > ~/.gmailctl/filters.xml to export filters into an xml file.
      4. click “Import filters”, select the xml file, and then click “Open file”.
      5. select filters to import, check “Apply new filters to existing email” and click “Create filters”.
         1. NOTE: selecting too many (~10) filters in one batch might cause it to stuck indefinitely. In that case, just refresh and retry.
         2. if the import fails, try to inspect the response with devtool/network. One error I encountered was caused by me disabled the “smart label” feature but included one category: "updates" in my rules.
3. back up your .gmailctl folder just like any other dotfiles.