System operational

Past Incidents

3rd March 2017

No incidents reported

2nd March 2017

No incidents reported

1st March 2017

No incidents reported

28th February 2017

No incidents reported

27th February 2017

No incidents reported

26th February 2017

No incidents reported

25th February 2017

No incidents reported

24th February 2017

Servers ISP Power outage

At approx 11:40 am EST our ISP (Cyberwurx) had power outage. UPS held machines up for about 20 minutes, but after that - all power was gone, and everything halted. At approx 12:30 am EST power was restored, machines started booting, but one rack was off the grid.

Sysadmin team is working on getting everything up, checking replication status and sorting all kinds of issues. No ETA on recovery.

EDIT: at approx 6 pm EST all servers and services have been restored and everything is operational. We'll need to revisit every single thing to confirm that everything is up and working, but so far - everything that's crucial for normal operation is alright.

Servers Server failure

One of physical servers had a failure this morning. Given machine is a host for multiple virtual machines for DatingVIP, and this caused misfunctionality for most of services, but mostly:

  • inbox
  • sessions handling
  • database access
  • memcache cache
  • redis storage
  • background processing

All virtual machines were restored within 1 hour, services are up and running, but piled up work has to be done so you might be seeing some delay in seeing stats, reports, etc.

Sorry for any inconvenience.