Press "Enter" to skip to content

Richy's Random Ramblings

ActivityPub for WordPress – How to fix ModSecurity to make it work

Like many people at the moment (due to Elon Musk’s purchase of Twitter), I’m moving from my nearly 14 year old Twitter account @rbairwell to Mastodon where I’m currently at @rbairwell@mastodon.org.uk . I was also pointed towards @pfefferle@mastodon.social‘s WordPress plugin ActivityPub For WordPress which allows me to put my blog directly “on the Fediverse” and allow you to follow it at @richyb@blog.rac.me.uk .

Symptoms / stuck on “Withdraw follow request”

However, after installing it the plugin and then trying to follow my blog, I just got a “Withdraw follow request” prompt in Mastodon – and, even after giving it a few minutes to account for server lag, my follow didn’t show up in WordPress->Users->Followers (Fediverse). If you want, you can just skip to the solution for root users .

Investigation / Mod Security Logs

My initial thought was that it was mod_security (a web-application firewall for the web site) which might be intercepting and blocking the request for security purposes. Turns out I was correct first time! Looking at my cPanel WHM's Security Center->ModSecurity Tools->Hits List, I found out that the requests were being blocked by rule 920420 of the OWASP Core Ruleset which was causing the following messages:

FieldData
Rule id920420: Request content type is not allowed by policy
SeverityCritical
Status403
RequestPOST /wp-json/activitypub/1.0/users/3/inbox
Action DescriptionWarning.
JustificationMatch of “within %{tx.allowed_request_content_type}” against “TX:content_type” required.
Details of the mod_security hit

Searching the mod security audit log for the request URL using grep /wp-json/activitypub/ /var/log/apache2/modsec_audit.log gave me the “incident id/file location”:

blog.rac.me.uk xxx.xxx.xxx.xxx - - [xx/xxx/xxxx:xx:xx:xx +0000] "POST /wp-json/activitypub/1.0/users/3/inbox HTTP/1.1" 403 4077 "-" "-" Y2z65HnPJZ2EEJpVH6GcggAAAA8 "-" /xxxxx/20221110/20221110-1321/20221110-132140-Y2z65HnPJZ2EEJpVH6GcggAAAA8 0 5109 md5:39bb07d5be0cc904943570b3a39fddbc

looking at /var/log/apache2/modsec_audit/xxxxx/20221110/20221110-1321/20221110-132140-Y2z65HnPJZ2EEJpVH6GcggAAAA8 showed me

...
--daee5752-B--
POST /wp-json/activitypub/1.0/users/3/inbox HTTP/1.1
Host: blog.rac.me.uk
...
Content-Type: application/activity+json
...
--daee5752-H--
...
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client xxx.xxx.xxx.xxx] ModSecurity: Warning. Match of "within %{tx.allowed_request_content_type}" against "TX:content_type" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "956"] [id "920420"] [msg "Request content type is not allowed by policy"] [data "|application/activity+json|"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/255/153"] [tag "PCI/12.1"] [hostname "blog.rac.me.uk"] [uri "/wp-json/activitypub/1.0/users/3/inbox"] [unique_id "Y2z65HnPJZ2EEJpVH6GcggAAAA8"]

Showing me that the ActivityPub protocol makes requests using the Content-type of application/activity+json which isn’t normally allowed with the OWASP Core Ruleset (OWASP CRS/3.3.2).

So how to fix this?

If you do not have root accessto your server, you might just have the option to turn off mod_security totally for your domain which will restore access.

If you do have root access, you’ll be able to view rule 92040in either your control panel (WHM users->Security Center->ModSecurity Tools->Rules List) or in your server at the listed path ( /etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf ). However, you’ll find that it lists:

# In case Content-Type header can be parsed, check the mime-type against
# the policy defined in the 'allowed_request_content_type' variable.
# To change your policy, edit crs-setup.conf and activate rule 900220.
SecRule REQUEST_HEADERS:Content-Type "@rx ^[^;\s]+" \
    "id:1,\
    phase:2,\
    block,\
    capture,\
    t:none,\
    msg:'Request content type is not allowed by policy',\
    logdata:'%{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-protocol',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/255/153',\
    tag:'PCI/12.1',\
    ver:'OWASP_CRS/3.3.2',\
    severity:'CRITICAL',\
    setvar:'tx.content_type=|%{tx.0}|',\
    chain"
    SecRule TX:content_type "!@within %{tx.allowed_request_content_type}" \
        "t:lowercase,\
        setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

But not the list of actually content-types allowed. Whilst these are defined in rule 901162 (found by searching for “tx.allowed_request_content_type“), you shouldn’t really modify the “vendor supplied rules”.

it’s best to add your own rule 900220 which is within crs-setup.conf. But it’s not advisable to change that file (in /etc/apache2/conf.d/modsec_vendor_configs/OWASP3/crs-setup.conf on my cPanel server) on cPanel servers as it might get updated/changed by cPanel itself.

Adding the new mod security rule to allow application/activity+json

Therefore, I’ve just created a new rule within mod_security (again WHM->Security Center->ModSecurity Tools->Rules List->Add Rule ) to match it with the additional content type listed:

SecAction \
 "id:900220,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_request_content_type=|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/octet-stream| |application/csp-report| |application/xss-auditor-report| |text/plain| |application/activity+json|'"

Note that the list of content types are separated by spaces, but are actually each enclosed by the pipe symbol – the pipe ( | ) isn’t the separator!

I deployed and restarted Apache and tried to follow myself again, and it all started working (and about 2 minutes after I posted this, it showed up in my timeline)

Hope it helps somebody else!

DNSSec signed Google Apps/G Suite Email

I’ve been using Google Apps, aka Google Workspace aka Google Suite (or just G Suite) for a while now and it’s annoyed me that I was getting “marked down” on e-mail security testers such as Internet.nl and the UK Government’s National Cyber Security Centre (NCSC) Check Your Email Security Service because Gmail for Business (G Suite) didn’t support DNSSEC (Domain Name System Security Extensions) signed MX hosts.

However, I’ve managed to find Google’s DNS Sec settings which – combined with other setups on my main domains – mean I get 4 green ticks from the NCSC, 97% from Internet.nl (I’m let down by Google’s support of old TLS and Ciphers settings and no DANE TLSA records) and all green (apart from DANE) on Hardenize : so nice strong secure email!

Read more: DNSSec signed Google Apps/G Suite Email

Google normally suggest you use the following MX (Mail Exchanger) records in your DNS settings if you use G Suite:

PriorityMail Server (MX Entry)
1ASPMX.L.GOOGLE.COM
5ALT1.ASPMX.L.GOOGLE.COM
5ALT2.ASPMX.L.GOOGLE.COM
10ALT3.ASPMX.L.GOOGLE.COM
10ALT4.ASPMX.L.GOOGLE.COM
The normal suggest Google Suite Email Servers for Businesses

(The records can actually be in any order and the priority can be anything – but Google do recommend that aspmx.l.google.com is set as the “highest priority” which is actually 1)

However, after a bit of searching (using DuckDuckGo and not Google 😉 ), led me to a blog post by Nis Bornoe and Kura the following G Suite DNSSEC signed MX records:

Google’s DNSSEC Signed Mail Servers (MX Entry)
mx1.smtp.goog
mx2.smtp.goog
mx3.smtp.goog
mx4.smtp.goog
Google’s “hidden” DNS SEC Signed MX Records

These domains are hosted on Google owned Charleston Road Registry (CRR)’s .goog top level domain (not to be confused with their .google and .gle brand top-level domains: or the 98 other ones they applied for) and .goog domains can “only be registered to Google Inc and its affiliates” so you’ve got some confidence they are legitimate.

However, whilst myself and Nis and Kura do not seem to have had any problems using these IPv4 and IPv6 supported DNSSEC signed nameservers (and according to DNSlytics and WhoisXMLAPI there are over 930 domains currently using them), they are not officially supported or documented (from what we can find) and have been running since at least 2019 – so they should be reasonably safe to use.

The only “catch” may may be that, for some reason, they do NOT have a reverse DNS (Pointer aka PTR) record setup – which is actually only a problem if those mail servers are use for sending OUT email (not just receiving it) – however, many only testers do assume that your inbound and outbound mail servers are the same. I can confirm, via a test email, that outbound mail goes out via servers such as mail-wr1-f52.google.com which are correctly configured.

Techy: Removing an rate limit block from Exim

During testing/investigation of an issue, I had to send “bad emails” to my Exim based mail server which after a number of attempts blocked me using the rate limit that had been configured:

451-The server has reached its limit for processing requests from your host.
451 Please try again later.

I didn’t want to wait for over an hour for the block to clear so I had to find out how to clear the block manually. On the mail server, I found the Exim data files in /var/spool/exim/db and using the exim_dumpdb tool to view the “ratelimit” block list:

# exim_dumpdb /var/spool/exim/ ratelimit

and then limiting it to the sending IP address (which I had checked via /var/log/exim_rejectlog ):

# exim_dumpdb /var/spool/exim/ ratelimit | grep 2001:db8:9:a::
28-Oct-2022 16:50:01.247104 rate:      1.877 key: 1h/per_mail/2001:db8:9:a::
28-Oct-2022 16:50:14.986765 rate:      1.937 key: 1h/per_conn//2001:db8:9:a::

Okay, that’s confirmed the block – but now to remove it. To do this, I had to use the “practically no documentation and no ui” tool exim_fixdb :

exim_fixdb /var/spool/exim/ ratelimit

It then just showed a prompt “>” with no instructions. But if the “key” was provided:

# exim_fixdb /var/spool/exim/ ratelimit
Modifying Exim hints database /var/spool/exim//db/ratelimit
> 1h/per_mail/2001:db8:9:a::
28-Oct-2022 16:50:01
0 time stamp: 28-Oct-2022 16:50:01
1 fract. time: .247104
2 sender rate: 1.877

Pressing “d” then deleted that record (and I repeated it for the other entry). Once I had finished, I just used “q” to quit exim_fixdb.

Hope it helps!

Techy: Changing Python Shebang path from absolute to env

I’m just working in some Python code, and quite a lot of it has the “shebang” line for the Python interpreter set as an absolute path (such as /usr/bin/python) which won’t work for me as I need the scripts to use whichever version of Python I have configured in my “environment”: and the ideal way to do this is to call is using /usr/bin/env python – so how do I change these…

cPanel: Disabling cPanel Store Promotions

A few people have got a bit annoyed at the promotions/advertisements within their WHM webhosting control panel as developed by cPanel Inc (specifically, I saw the feature request “give server admins a way to turn off spam ads“).

Whilst cPanel partners (i.e. whoever your brought your cPanel licence from – usually your datacenter or web hosting provider) have the ability to toggle these cPanel Store Purchases from their cPanel Inc “manage2” interface, there is a way of disabling it yourself by “abusing” the internal testing code built into cPanel (specifically /usr/local/cpanel/Cpanel/Config/Sources.pm which is called by /usr/local/cpanel/Cpanel/Whostmgr/Store.pm which, in turn, is called by /usr/local/cpanel/Whostmgr/Store/Product/ImunifyAVPlus.pm ):