Press "Enter" to skip to content

Tag: cpanel

ActivityPub for WordPress – How to fix ModSecurity to make it work

Like many people at the moment (due to Elon Musk’s purchase of Twitter), I’m moving from my nearly 14 year old Twitter account @rbairwell to Mastodon where I’m currently at @rbairwell@mastodon.org.uk . I was also pointed towards @pfefferle@mastodon.social‘s WordPress plugin ActivityPub For WordPress which allows me to put my blog directly “on the Fediverse” and allow you to follow it at @richyb@blog.rac.me.uk .

Symptoms / stuck on “Withdraw follow request”

However, after installing it the plugin and then trying to follow my blog, I just got a “Withdraw follow request” prompt in Mastodon – and, even after giving it a few minutes to account for server lag, my follow didn’t show up in WordPress->Users->Followers (Fediverse). If you want, you can just skip to the solution for root users .

Investigation / Mod Security Logs

My initial thought was that it was mod_security (a web-application firewall for the web site) which might be intercepting and blocking the request for security purposes. Turns out I was correct first time! Looking at my cPanel WHM's Security Center->ModSecurity Tools->Hits List, I found out that the requests were being blocked by rule 920420 of the OWASP Core Ruleset which was causing the following messages:

FieldData
Rule id920420: Request content type is not allowed by policy
SeverityCritical
Status403
RequestPOST /wp-json/activitypub/1.0/users/3/inbox
Action DescriptionWarning.
JustificationMatch of “within %{tx.allowed_request_content_type}” against “TX:content_type” required.
Details of the mod_security hit

Searching the mod security audit log for the request URL using grep /wp-json/activitypub/ /var/log/apache2/modsec_audit.log gave me the “incident id/file location”:

blog.rac.me.uk xxx.xxx.xxx.xxx - - [xx/xxx/xxxx:xx:xx:xx +0000] "POST /wp-json/activitypub/1.0/users/3/inbox HTTP/1.1" 403 4077 "-" "-" Y2z65HnPJZ2EEJpVH6GcggAAAA8 "-" /xxxxx/20221110/20221110-1321/20221110-132140-Y2z65HnPJZ2EEJpVH6GcggAAAA8 0 5109 md5:39bb07d5be0cc904943570b3a39fddbc

looking at /var/log/apache2/modsec_audit/xxxxx/20221110/20221110-1321/20221110-132140-Y2z65HnPJZ2EEJpVH6GcggAAAA8 showed me

...
--daee5752-B--
POST /wp-json/activitypub/1.0/users/3/inbox HTTP/1.1
Host: blog.rac.me.uk
...
Content-Type: application/activity+json
...
--daee5752-H--
...
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client xxx.xxx.xxx.xxx] ModSecurity: Warning. Match of "within %{tx.allowed_request_content_type}" against "TX:content_type" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "956"] [id "920420"] [msg "Request content type is not allowed by policy"] [data "|application/activity+json|"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/255/153"] [tag "PCI/12.1"] [hostname "blog.rac.me.uk"] [uri "/wp-json/activitypub/1.0/users/3/inbox"] [unique_id "Y2z65HnPJZ2EEJpVH6GcggAAAA8"]

Showing me that the ActivityPub protocol makes requests using the Content-type of application/activity+json which isn’t normally allowed with the OWASP Core Ruleset (OWASP CRS/3.3.2).

So how to fix this?

If you do not have root accessto your server, you might just have the option to turn off mod_security totally for your domain which will restore access.

If you do have root access, you’ll be able to view rule 92040in either your control panel (WHM users->Security Center->ModSecurity Tools->Rules List) or in your server at the listed path ( /etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf ). However, you’ll find that it lists:

# In case Content-Type header can be parsed, check the mime-type against
# the policy defined in the 'allowed_request_content_type' variable.
# To change your policy, edit crs-setup.conf and activate rule 900220.
SecRule REQUEST_HEADERS:Content-Type "@rx ^[^;\s]+" \
    "id:1,\
    phase:2,\
    block,\
    capture,\
    t:none,\
    msg:'Request content type is not allowed by policy',\
    logdata:'%{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-protocol',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/255/153',\
    tag:'PCI/12.1',\
    ver:'OWASP_CRS/3.3.2',\
    severity:'CRITICAL',\
    setvar:'tx.content_type=|%{tx.0}|',\
    chain"
    SecRule TX:content_type "!@within %{tx.allowed_request_content_type}" \
        "t:lowercase,\
        setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

But not the list of actually content-types allowed. Whilst these are defined in rule 901162 (found by searching for “tx.allowed_request_content_type“), you shouldn’t really modify the “vendor supplied rules”.

it’s best to add your own rule 900220 which is within crs-setup.conf. But it’s not advisable to change that file (in /etc/apache2/conf.d/modsec_vendor_configs/OWASP3/crs-setup.conf on my cPanel server) on cPanel servers as it might get updated/changed by cPanel itself.

Adding the new mod security rule to allow application/activity+json

Therefore, I’ve just created a new rule within mod_security (again WHM->Security Center->ModSecurity Tools->Rules List->Add Rule ) to match it with the additional content type listed:

SecAction \
 "id:900220,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_request_content_type=|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/octet-stream| |application/csp-report| |application/xss-auditor-report| |text/plain| |application/activity+json|'"

Note that the list of content types are separated by spaces, but are actually each enclosed by the pipe symbol – the pipe ( | ) isn’t the separator!

I deployed and restarted Apache and tried to follow myself again, and it all started working (and about 2 minutes after I posted this, it showed up in my timeline)

Hope it helps somebody else!

Techy: Removing an rate limit block from Exim

During testing/investigation of an issue, I had to send “bad emails” to my Exim based mail server which after a number of attempts blocked me using the rate limit that had been configured:

451-The server has reached its limit for processing requests from your host.
451 Please try again later.

I didn’t want to wait for over an hour for the block to clear so I had to find out how to clear the block manually. On the mail server, I found the Exim data files in /var/spool/exim/db and using the exim_dumpdb tool to view the “ratelimit” block list:

# exim_dumpdb /var/spool/exim/ ratelimit

and then limiting it to the sending IP address (which I had checked via /var/log/exim_rejectlog ):

# exim_dumpdb /var/spool/exim/ ratelimit | grep 2001:db8:9:a::
28-Oct-2022 16:50:01.247104 rate:      1.877 key: 1h/per_mail/2001:db8:9:a::
28-Oct-2022 16:50:14.986765 rate:      1.937 key: 1h/per_conn//2001:db8:9:a::

Okay, that’s confirmed the block – but now to remove it. To do this, I had to use the “practically no documentation and no ui” tool exim_fixdb :

exim_fixdb /var/spool/exim/ ratelimit

It then just showed a prompt “>” with no instructions. But if the “key” was provided:

# exim_fixdb /var/spool/exim/ ratelimit
Modifying Exim hints database /var/spool/exim//db/ratelimit
> 1h/per_mail/2001:db8:9:a::
28-Oct-2022 16:50:01
0 time stamp: 28-Oct-2022 16:50:01
1 fract. time: .247104
2 sender rate: 1.877

Pressing “d” then deleted that record (and I repeated it for the other entry). Once I had finished, I just used “q” to quit exim_fixdb.

Hope it helps!

cPanel: Disabling cPanel Store Promotions

A few people have got a bit annoyed at the promotions/advertisements within their WHM webhosting control panel as developed by cPanel Inc (specifically, I saw the feature request “give server admins a way to turn off spam ads“).

Whilst cPanel partners (i.e. whoever your brought your cPanel licence from – usually your datacenter or web hosting provider) have the ability to toggle these cPanel Store Purchases from their cPanel Inc “manage2” interface, there is a way of disabling it yourself by “abusing” the internal testing code built into cPanel (specifically /usr/local/cpanel/Cpanel/Config/Sources.pm which is called by /usr/local/cpanel/Cpanel/Whostmgr/Store.pm which, in turn, is called by /usr/local/cpanel/Whostmgr/Store/Product/ImunifyAVPlus.pm ):

cPanel: Unable to upgrade from MySQL to MariaDB

Due to changes in the way MariaDB has setup their Centos yum repositories, cPanel (at the time of writing) is unable to access the necessary files to upgrade people from MySQL to MariaDB (as it needs to do it in stages and MariaDB 10.0 is no longer available).

Whilst the cPanel knowledgebase article entitled “Can’t upgrade from MySQL 5.6 to MariaDB” and cPanel have acknowledged it as an internal bug with the reference CPANEL-40434, they say there is “no workaround at this time”. This is incorrect!

As I posted on their forums in the “One of the configured repositories failed (MariaDB106)” thread (unfortunately my post – 14 hours on – is still pending moderation), there is a reasonably quick fix server administrators can do. Here’s the full text of the post “just in case”:

Since I’ve hit this problem on a new server migrating from MySQL to MariaDB (and my much earlier post about how to fix the general issue seems to have just expired rather than be approved), here’s what I did to fix it.

Platform: Centos v7.9 on x64 – migrating from MySQL 5.7 to MariaDB 10.6

If you do not have /etc/yum.repos.d/MariaDB102.repo , start an upgrade to MariaDB and let it die.

1. Copy /etc/yum.repos.d/MariaDB102.repo to /etc/yum.repos.d/MariaDB102-patch.repo
2. Modify /etc/yum.repos.d/MariaDB102-patch.repo to change the baseurl to https://archive.mariadb.org/mariadb-10.2/yum/centos7-amd64
3. Copy that file now as /etc/yum.repos.d/MariaDB103-patch.repo
4. Modify MariaDB103-patch.repo so “102” is changed to “103” throughout and the base URL has 10.3
5. Repeat steps 3 and 4 to go from “103/10.3” to “104/10.4” etc all the way up to 106/10.6
6. Do yum clean all;yum makecache
7. Go to the Database upgrade page (select “Ignore current upgrade” if necessary)
8. And upgrade!

Bit long winded and fiddly, but it worked for me(tm)

Bug Report: [WontFix] “Market providers” errors when cPanel store disabled by license holder

In an effort to prove to myself that I am actually trying to do work this last month, I’m making a note of all the bugs in 3rd party software I find.

Today is a bug reported to cPanel Inc on the 11th June 2022 under their tracking request ID 94455138 which caused a nasty looking error message (and big log dumps in log files) in their WHM interface of their cPanel control panel suite when certain sections were accessed.