Mailroom

v1.0.0

2 hours ago

1. • First release. A newsletter for any Grav 2 site, with no store behind it: subscribers with a real consent record, lists people can leave one at a time, double opt-in, branded campaigns with subject tests, segments, automations, CSV import and export, provider webhooks for bounces and complaints, deliverability checks, and campaign and audience reports
   • Its own database, set up by itself. SQLite by default at user-data://mailroom/mailroom.sqlite, created and migrated on first use. MySQL and PostgreSQL are supported too, migrated with bin/plugin mailroom migrate (or by themselves with database.auto_migrate: auto). Every table is named mailroom_*. The remembered "schema is up to date" answer is kept against the engine's migration definitions as well as Mailroom's wrappers, so an update that adds a step to one is always looked for. A table a later migration adds is looked for before it is used, so on MySQL or PostgreSQL an unsubscribe, a signup or a send keeps working, and one line in the log says so, until somebody runs the migrate command. While migrations are waiting, the public routes answer 503 with Retry-After rather than an error
   • Public pages under a route base of your choosing, /newsletter by default (and when the setting is left empty, since pages at the site root would sit in front of the site's own): subscribe, confirm, unsubscribe, the preference center, the web view, the open pixel, the click redirect and provider webhooks. The confirm, unsubscribe, preference and message pages extend the theme's base template (pages.base_template, partials/base.html.twig when empty) and carry a small neutral stylesheet that can be switched off. They talk about a site, never a store or an account: the unsubscribe page says only that the emails stop
   • A signup box anywhere: {{ mailroom_form() }} in a theme or a page, or the Form plugin's mailroom action on a form of your own. Signups through the Form plugin count in the same per-visitor limit as the signup route (rate_limits.subscribe, 10 an hour). {{ mailroom_form('newsletter', {lists: ['security', 'events']}) }} offers several lists as checkboxes, under their own names and none checked for them, and signs the person up to the ones they check; the form will not send with none. Each list keeps its own double opt-in and one confirmation email covers them all. A signup through the box, with JavaScript or without, is recorded as form; JSON from elsewhere is api, and a caller may say which with the route's source field. A form's own consent_text is the sentence recorded for the people who sign up through it, carried back signed with the site secret. Mailroom's small stylesheet is on every front-end page (switch it off with pages.builtin_css), so the box is styled wherever it is, with square checkboxes
   • Branded email layout. Every campaign, confirmation and automation email is wrapped in one layout with your logo, accent, background and text colors, footer text and postal address (branding.*). A campaign set to the plain layout, and the text part of every message, still carry the footer text and the postal address. The accent is left empty by default: email is then teal, and the public pages and the signup box take the theme's own accent where it has one (Quark2's), so they match the site
   • A worker that runs itself. bin/plugin mailroom work runs every minute from Grav's scheduler, and the jobs a request queued (a confirmation email after a signup) run straight after its response, so mail goes out before cron is even set up. Daily housekeeping sweeps finished jobs, old tracking rows, stale uploads and the mail log on the retention settings
   • status says when the scheduler is not running. bin/plugin mailroom status no longer calls a site healthy when the worker has not run for more than 15 minutes (or never has), even with nothing waiting, because that means Grav's scheduler is not being called. The warning says to add the scheduler to the crontab and prints the exact line for the site, built by Grav itself the way bin/grav scheduler --install builds it: * * * * * cd <site root>;<php> bin/grav scheduler 1>> /dev/null 2>&1. A confirmation sent by the inline drain after a signup does not count as the worker running
   • Its own mail log. Every message Mailroom hands the Email plugin is logged with who, what, when, and its first open and bounce, and a job that runs twice never sends the same message twice. Addresses in it are kept lower-cased, the way the engine keeps every address, so looking up what one address was sent reads the index
   • Admin API and MCP tools. Every admin screen is a route under /mailroom, every route is an MCP tool (mailroom_*), with three permissions: mailroom.view, mailroom.manage and mailroom.send. POST /mailroom/erase and bin/plugin mailroom erase forget everything about one address, keeping a suppression on it as a hash with the address removed
   • Sends as the Email plugin until you say otherwise. Mailroom's own From name and address are an override: left empty, mail goes out as the Email plugin's From, and the "no From address" banner, the status line and the Health check speak up only when neither has one
   • Dashboard banners in Admin Next for mail that cannot leave, a campaign that stopped itself, no From address anywhere, a transport that strips the unsubscribe header, a segment that could not be counted, a failing deliverability check, a high bounce rate, and webhooks that went quiet or have no secret
   • Command line: migrate, status, work, erase, import:csv, export:csv, segments:count, segments:preview, deliverability:check, webhook:replay and seed, all under bin/plugin mailroom
   • A demo to look around in. bin/plugin mailroom seed fills an empty Mailroom with three months of a small project's newsletter: lists Newsletter, Product updates and Events, tags for people who signed up on the site, came to a webinar or try every release early, four hundred people who joined through the signup form, an import, the admin or the API, seven campaigns with real numbers, four segments and two automations (a welcome series on a confirmed signup, and one on a tag). Nothing is sent, and seed --purge removes exactly what it added
   • A Welcome series of its own. The ready-made Welcome series on the automations screen is two emails about the site (a welcome that says what you send and how often, and a few good places to start three days later), not a shop's, with no purchase exit
   • Mailroom's own names throughout. The admin's heading is Mailroom, and nothing on a Mailroom site talks about a store: no customer record on a subscriber, no checkout or account in the ways somebody joined, and automations that start on a signup, a confirmation or a tag. The signup form's ids are mailroom-1, mailroom-1-email and so on
   • An admin inside Admin Next. Mailroom's own page at /plugin/mailroom in the sidebar, with every screen on its own address (#/campaigns/12, #/settings/sending) so a reload or Back lands where you were: the overview, campaigns and their editor, subscribers, the import, audience and segments, automations, templates, suppressions, reports, health and settings. It is drawn with Admin Next's own colors, borders and controls, so it follows the light and dark themes, and asks before leaving an editor with changes that have not been saved
   • Settings on the Settings screen. The plugin's settings form is drawn inside Mailroom's page, and the admin's own Configure button and /plugins/mailroom go there. The mail transport, queue worker and provider cards on it read Mailroom's own API
   • Trusted proxies. Consent records and rate limits use the visitor's address, read from X-Forwarded-For only when the request came from a proxy listed in security.trusted_proxies, with a port a proxy wrote on a hop (203.0.113.9:5678, [2001:db8::1]:443) taken off. An IPv6 visitor is rate limited by their /64, so one connection cannot take a fresh budget from every address it was handed. Rate-limit keys are stored hashed
   • Subscribers in bulk, and every list on a person. The Subscribers table has a checkbox per row, one for the page and "Select all N matching these filters", with a bar that adds people to a list or takes them off one, adds or removes a tag, unsubscribes or deletes them (POST /mailroom/subscribers/bulk, mailroom.manage, MCP tool mailroom_bulk_subscribers). A subscriber's page shows every list with where they stand on it, an Add to list or Remove beside each, and their tags as chips with a box to add one. A list on the Audience tab opens its own page with its settings, Edit, "Add people" (paste addresses, with names if you like) and its people, and a list or tag count opens the Subscribers table filtered to it, with the filter in its picker and in the address
   • Adding somebody to a list asks how. Every add from the admin ("Add somebody", "Add to list" on a person's page, the bulk bar, a list's "Add people", and the MCP tools mailroom_add_subscriber and mailroom_bulk_subscribers) offers "Send them an invite to confirm" (the ordinary confirmation email; they wait on that list until they press it, and have no consent date until they do) or "Add as subscribed, they already agreed" with a note saying where, kept in their consent history. The default follows the list's "Ask people to confirm". Suppressed addresses are never added, and nobody who left a list, or every list, themselves is put back on it: not by an admin's add, not by the list boxes on their page, and not by an automation's "add to list" step, which skips them and says so in the log. How somebody left stays theirs when an admin later unsubscribes or suppresses them too, and their consent history is read as well. "Put them back on" is the way back for somebody who asked, and it needs a note saying how they asked. A confirmation page for an invite they never asked for (an admin's, or an import's) waits for them to press the button rather than pressing it itself
   • A consent history for every person. Every change to somebody's consent is a row in mailroom_consent_events (migration 0014_consent_events): signing up, confirming, an admin's add or invite with its note, leaving (by the link in a campaign, their mail client's button, the preference center, an admin, a bounce or a complaint), coming back, an import with its file, and an automation putting them on a list, with the admin who did it or the address and browser a person's own change came from. It is kept for as long as they are, drawn under Consent on their page (the newest eight, the rest behind "Show N earlier"), answered by GET /mailroom/subscribers/{id} and the mailroom_get_subscriber tool, and erased with them: bin/plugin mailroom erase and POST /mailroom/erase count it as consent history rows
   • A campaign can go to several lists, or to every list with a segment. Pick the first list in "Send it to" and check the others under "Also send to", or choose "All lists, with a segment" to mail everybody on any list who matches it. Somebody on two of the lists gets one email, and the count under the pickers is worked out the same way. The unsubscribe link, and the unsubscribe button in the mail client, take them off whichever of the campaign's lists they are on and leave their other lists alone. The campaign's page says "Sent to: Newsletter, Events" (or "All lists, matching" the segment), and the performance report names the lists too. A campaign to one list works exactly as before. Migration 0015_campaign_lists adds mailroom_campaign_lists
   • Export one person's data. A subscriber's page has "Export their data", which downloads everything Mailroom holds about that person as <address>-data.json: their details, every list with where they stand on it and when, their tags, the whole consent history, every message they were sent with when it was opened and each link they pressed, their automation walks, what the mail provider reported about them (deliveries, bounces, complaints), any suppression on their address, the import that brought them in, and every row in Mailroom's message log addressed to them. It holds everything an erasure deletes about them, and leaves out their signing secret and anybody else's details. The same document is GET /mailroom/subscribers/{id}/export and the MCP tool mailroom_export_subscriber_data, both needing mailroom.manage, like the CSV export
   • {{ site.name }} and {{ site.url }} merge tags. The editors' lists of merge tags offer them for the site's title and address, and the demo seed writes its copy with them. {{ store.name }} and {{ store.url }} still work and render the same words
   • The segment editor files the subscriber's own fields under Subscriber, rather than under the engine's "Newsletters"
   • Form dialogs name their button. The page hands the admin's form dialog the section's confirmLabel as its submitLabel, so "Add somebody", "Save" and the rest read on the button rather than the admin's default
   • Links in email come from the site's settings, never from a visitor's request. Mailroom has its own Site URL (site_url, on the Public Pages tab), read before Grav's system.custom_base_url. A request's Host header is anybody's to write, so a signup sent with somebody else's Host no longer decides where its confirmation link points. With neither setting, only an admin's own request (an account with mailroom.manage, or a super user) may name the site, and an email the worker would otherwise build with links that go nowhere waits on the queue, costing none of its attempts, until one of the two is set. A dashboard banner, a status line and a Health check say so
   • The Site URL fills itself in. With neither Mailroom's Site URL nor Grav's system.custom_base_url set, the first time an admin who can manage Mailroom (or a super user) opens the Admin Next dashboard or Mailroom's page, the address they came in on is written into site_url, the same way Mailroom's settings are saved, with one line in Grav's log. Only over HTTPS, or plain HTTP on a local site (localhost, .test, .local, a private address), never from the CLI, never from anybody else's request and never over a value that is already there. So most sites never see the banner, and the emails held waiting for it go out on the next worker run rather than sitting out the rest of their wait
   • Provider reports without the address. What a mail provider's webhook said about a message (a delivery, a bounce, a complaint) is kept with the address's hash and without the address itself, in the payload or in the provider's reason text, and an erasure (bin/plugin mailroom erase, POST /mailroom/erase) or the admin's Delete takes them with the person, counted as provider reports. Migration 0016_event_addresses takes the address out of the reports already stored
   • A mail client's unsubscribe button always works. Its request comes from the mail provider's servers, so one with a valid link is not counted against that address's hourly limit; one with a link that does not check out is counted as usual
   • The Health tab watches the worker. Its Transport group has a queue worker line that warns when the worker has not run for 15 minutes while jobs are waiting, from the same run mark status reads, and says to run bin/plugin mailroom status for the crontab line
   • Imports that say what they will do. The import screen shows, before you press Start, how many people would be new, already here, skipped and suppressed against the list you picked, and why; a file headed plain email (or Name, First name and the like) needs no column picked; "Where did these people agree?" (basis on the API, --basis on import:csv) is kept in each person's consent history, the file name when it is empty; and nobody who left a list, or every list, by their own hand is put back by an import
   • A provider's retried webhook counts once. Every provider event carries a key of its own (migration 0017_event_keys), so a webhook the provider sends again, or one replayed with webhook:replay, is recorded once and a single temporary failure is never counted as three soft bounces
   • A signup's answer appears where the person is looking. When the box takes a signup, its fields and button give way to the answer in place, and the answer takes the focus so a screen reader reads it out (it was printed under the button, often out of view). A refusal scrolls into view and leaves the fields to correct
   • The admin reads as one page under its header. The screens under Mailroom's header no longer leave an empty band where a second "Mailroom" heading used to be; the line that says what each screen is for sits right under the header. A table's Open and ⋯ stay on one line. #/health opens the Health tab, the same as #/deliverability
   • Built on the newsletter kit 1.0.3. An automation's trigger and tag steps name the tag ("When the tag Early adopter is added"), not its code. The preference center is "Preference center" in a subscriber's consent history and in how they joined or left a list. Report charts write their dates the way the rest of the admin does ("Jul 12"). The preference center says "Choose what you want to hear about." and the consent box asks you to check it, not tick it. Under "Sent to" a campaign names the lists it went to (or "of 298" while it is still sending) rather than repeating the number, an automation's Left card reads "Most often: marked it as spam", and a webhook address wraps where its line is full