Service Status

Security incident on Canterlot Avenue
Incident Report for Poniverse
Postmortem

There was a targeted attack at a Canterlot Avenue administrator, which hit Canterlot Avenue itself at 16:23 UTC: the attacker obtained the password this admin used to log into Canterlot Avenue as well as several other, unrelated services.

After the attacker obtained this admin's password, they used it to access the phpFox admin area and start handing out administrative privileges to a number of other unauthorized users. In addition, they also impersonated the admin to post numerous offensive messages under their name.

The attacker was able to use this administrator's account for about ten minutes until an opspony was notified of the event and was able to react by disabling the general public's access to canterlotavenue.com. Several of our techponies then spent most of Easter Monday auditing the site, poring over logs, and scouring our databases to piece together a sense of exactly how the attacker got in and what they did.

After attaining confidence that no vulnerabilities in phpFox or our infrastructure were used to carry out this attack, and after cleaning up the damage that was done, we have decided it will be safe to bring Canterlot Avenue back online at some point in the next 24 hours as staff activities stabilize and a few of us have had time to get some sleep.

How we fixed it:

Several of our technical staff educated the admin in question about good password practices, two-factor authentication, and how to effectively use a password manager. They reset their passwords for Canterlot Avenue as well as all their other compromised accounts to unique, strong ones using this new knowledge. All posts that the attacker made while impersonating them were removed and an audit was performed of user permissions on the site to ensure that the correct users have the correct level of access.

We took complete snapshots of the database, site, and relevant access logs that will be retained for potential future analysis and internal recordkeeping purposes.

The incident will furthermore be used as a case study for training other staff with privileged access about how to password.

What the attacker did and didn't get:

The attacker had an administrator's Canterlot Avenue password. This granted them the ability to log in as this admin to interact with the website, impersonating them in the community, as well as unrestricted access to phpFox's web-based admin area. They were able to access users' names, email address, and IP addresses through the UI but there is no evidence of any attempt to mass-dump any data in there; all evidence points to the attacker having gone in with the sole intent of promoting and demoting a number of unauthorized, specific, and predetermined users.

There is no evidence of any software bugs or configuration vulnerabilities having played a role; the attacker did not employ any exploits against phpFox or our server infrastructure, and there is no evidence that they gained any level of access to the servers themselves. Their access was restricted to what they could access through interacting with the Canterlot Avenue website itself, which isn't that much.

What this means for you:

The only personal information at risk are users' email addresses and IP addresses that the attacker may have seen while they were in the admin area. No user passwords are at risk - all our users' password hashes are stored in Poniverse.net's system, and there is no way to access them from phpFox's admin area. For that reason, we will not be forcing any password resets.

However, we'll take the opportunity to remind or inform you that using separate passwords for every service, and using a password manager to keep track of them all, is a good idea that will protect you from becoming the victim of a similar attack. This could have happened to anyone.

Love, ~Lavo, Hazel, Tinker, Comp, GoldenHeartVA, & Feld0 Princess Pixel Wavelength's disciples who were involved in the response

Posted 4 months ago. Apr 03, 2018 - 16:01 PDT

Resolved
Canterlot Avenue is back online!
Posted 4 months ago. Apr 03, 2018 - 16:01 PDT
Identified
We have finished our investigations. Preparations are being made to bring the site back online.
Posted 4 months ago. Apr 02, 2018 - 21:47 PDT
Investigating
It appears a bad actor has breached Canterlot Avenue. The site will be offline while an investigation into what happened proceeds. There will be an update when it's done.

~PoniOps team
Posted 4 months ago. Apr 02, 2018 - 09:43 PDT