We’re sorry

We understand that the outage that occurred on the night of March 27 (ACDT) affected many of our customers, and for that we are sorry. We will be in contact soon regarding what we can do to help.

‌

Cause

We can now reveal that the outage was caused by a faulty script run on Homer server that replaced the file permissions on critical infrastructure. This caused most of our interlinked software to malfunction, leading to the outage. The scripts that were being run were related to our up-and-coming Game Hosting infrastructure, which has been postponed indefinitely until we have migrated Homer Server (more info below).

‌

Is My Data Safe?

No data was compromised, or destroyed in this incident.

‌

Was This Related to a Hack?

No, the outage was caused by a faulty internal script run by Soularc.

‌

Next Steps

After consulting with Plesk professionals, the next steps are to migrate all data on Homer Server to a new Plesk installation. We are currently consulting with Plesk which will be the best way forward in terms of this migration. From your perspective, nothing will change after the migration. It will be the same, familiar web hosting control panel with all of your data still intact. The only thing that will be different will be our new server name (which we are excited to release soon!).

Alongside the launch of our new server, we are going to rebuild our backup infrastructure from the ground up to ensure that an incident like this cannot happen again1.

‌

I still have questions

If you have any questions whatsoever regarding this incident, please do not hesitate to contact us by visiting sending an email to hello@soularc.net

‌

1 Please refer to our SLA for what we do for you in terms of uptime guarantees.

Posted Mar 28, 2021 - 20:30 ACDT

Resolved

Webmail is now back online for all domains. We will provide more information regarding the cause of the outage tomorrow.

We are sorry for any inconveniences caused.

Posted Mar 27, 2021 - 21:55 ACDT

Update

Plesk support is currently investigating the issue regarding Webmail hosting.

Additionally, we are currently in contact with the Plesk disaster recovery team for support with migrating to a new Plesk installation. We currently believe that this transfer will occur sometime next week during business hours (due to timezone differences). However, it currently seems as though this migration will not incur in any downtime, this is not confirmed though.

Posted Mar 27, 2021 - 14:59 ACDT

Update

Additionally, webmail is still offline, however the vast majority of our clients utilise a mail client application (i.e. Outlook or Apple Mail).
If you are effected by the webmail outage, please contact us at hello@soularc.net

Posted Mar 27, 2021 - 09:50 ACDT

Update

We are continuing to monitor the situation regarding the mail server and shared web hosting.

In terms of future plans, we are investigating an option to migrate to a new server with the assistance of Plesk support. We will keep you update with information as it comes.

Posted Mar 27, 2021 - 09:47 ACDT

Update

Mail is now back online 🎉

A huge shoutout goes to the Plesk support team who helped resolve this issue.

Now, we are investigating the issue with Webmail and also planning the next steps for Homer Server, more info to come.

Posted Mar 27, 2021 - 07:41 ACDT

Update

We have referred the issue to Plesk professionals who are currently in the process of fixing our mail servers - this should only take 1-2 hours.
We are unsure when Webmail will be back online.

We are also investigating what will be the best post-incident response and will keep you updated.

Sorry for any inconveniences caused.

Posted Mar 27, 2021 - 02:08 ACDT

Monitoring

We have identified an issue with FastCGI PHP causing 503 errors with website. We have therefore transitioned all websites to PHP running as an FPM application.
Webmail is still experiencing an outage, and we are currently unsure regarding the status of SMTP and IMAP in addition to whether our mail server is currently receiving email.

Posted Mar 26, 2021 - 22:48 ACDT

Update

We can confirm that no data loss has occurred and that no data has been breached.

Posted Mar 26, 2021 - 21:12 ACDT

Identified

We have identified an issue with file permissions on Homer Server. We are currently restoring file permissions to resolve this issue.

Posted Mar 26, 2021 - 21:10 ACDT

Investigating

We are currently investigating the issue and will provide updates as they come.

Posted Mar 26, 2021 - 20:40 ACDT

This incident affected: Landing Page, Billing System, Shared Hosting Login, Shared Hosting Mail and Servers (Nelson Server).