VATUSA Website and Data Systems

Don Desfosse · « **Reply #15 on:** May 31, 2009, 08:04:15 AM »

Andrew,

To echo the other comments, you've done a phenomenal job in very short order. It is very, very much appreciated by the ARTCC staffs and the entire membership. A huge, huge THANK YOU!!

Thomas King · « **Reply #16 on:** May 31, 2009, 09:03:36 AM »

yes Thank You for everything you are doing.

Justin A. Martin · « **Reply #17 on:** May 31, 2009, 09:41:33 AM »

Andy,

G-d is testing how good you really are in these past 24 hours

You have done great things in so little time to get this back on the road. I appreciate all of your efforts, as do all of the VATUSA members I am sure, and I look forward to seeing your progress as USA6.

Your friend,

Andrew Podner · « **Reply #18 on:** May 31, 2009, 10:40:47 AM »

This is a mini update as of about 10:30am

Right Now I am working on some local custom server configuration setting to see if we can make it cooperate with our legacy systems. Trying to overhaul the certcenter to make it agree with the server is not working out real well. I am focusing on trying solutions that plug leaks quickly. Other than the new public site, none of these are permanent solutions, just patches so we can buy time to fix it right.

more to follow...........

Harold Rutila · « **Reply #19 on:** May 31, 2009, 10:57:18 AM »

Out of curiosity, do we have an idea of who might be responsible for this?

Alex Bailey · « **Reply #20 on:** May 31, 2009, 11:00:20 AM »

Quote from: Harold Rutila

Out of curiosity, do we have an idea of who might be responsible for this?

No idea at the moment, but it does look to be somewhat random although the attack was centered on an old inactive system running on our site. One would think that the hacker would need to know it was there, but I really don't know and Andrew can provide more insight than I can on this issue.

I do know that it took down the entire VATSIM.info server, including a couple other divisions (China and India to my knowledge).

Andrew Podner · « **Reply #21 on:** May 31, 2009, 12:06:42 PM »

UPDATE: SUNDAY 12:00pm

Even the best laid plans are subject to Murphy's law! But we have managed to bring another system back online, just not the one I wanted.

The bad news:
-Cert Center/Testing login is down, but it looks like testing and reassigns are working

The Good News:
-The instructor site / promotions panel is back online
-The ATM control panel is back online and functional as far as I can tell (if anyone accepts a transfer today let me know if it works.)
-All data and records are intact as far as I can tell
-The training center is still up and running
-Email is running
-New Member Application is working and the Basic Controller test is working
-Forums are working
-Everything that is broken can be fixed!

Cert Center Update:
Ok guys, for you code nerds (like me) out there, you'll understand what the problem is when I say that all forms and sessions in the cert center only work under a php4 type server setup. The config file I need to change is sitting in the directory just above us and is owned by VATSIM. I have submitted our case for having the config file modded so the cert center will start working. This does not mean that they will do it, only that I have asked.

For the rest of you, essentially the cert center is a very heavily modified software package from several years back and the problem is when you fix one thing, something else breaks, so before we commit to any further major work in the Certcenter, I would like to try to just get the server to agree with the software.

IF we get the server mod, then everything should start spinning again. If not, Staff will have to make a judgment on what move we want to make, whether to attempt a major salvage operation, or start anew. The good news is that we are down to the last major system that needs to be repaired. I will update again once I have heard from the powers above as to what the server config outcome will be.

Answer to the who did question
We know that the attack began on the 28th and the last hop into the system came from a system in Holland. I have not explored it beyond that, and probably will not until recovery is complete.

more to follow..........

Andrew Podner · « **Reply #22 on:** May 31, 2009, 03:47:12 PM »

3:30pm Update:

Cert Center: Still waiting to find out if we will be able to get the server reconfigured. I know that there is frustration about this and everyone who needs to take an exam is very anxious to do so. I promise you that the cert center is my #1 priority and we are doing all things possible to rectify the issue. Thank you for your patience.

Brandon Bartell · « **Reply #23 on:** May 31, 2009, 05:26:52 PM »

Andrew

I did a lot of work with the PHP Test. I made some modifications. When I was working on the Atlanta Site we used a version that was modified similar to the one VATUSA's uses. You may not have to modify that php config file.

If you want to talk more email me. brandon.bartell@g-m-x.com (remove the dashes). Maybe when can get on teamspeak I have some ideas.

Brandon

Andrew Podner · « **Reply #24 on:** May 31, 2009, 06:11:06 PM »

6:00pm Update:

Brandon, I just saw your post, thank you for the offer. I may contact you shortly.

Myself, Michael (USA3) and Rob (USA9) have been collaborating for the last few hours on the cert center and I am pleased to report that we have overcome the worst of the obstacles which was fixing the authentication system. At this time we are going to break for dinner and then come back to working on the cert center. Essentially what is left to be done is to update the form pages to start working right. This is a fix that we have already performed several times in other systems, and I feel like we will be able to overcome most of the problems with relative ease.

This is a very good development on two fronts, first that we are getting the system back online, and second, that this is coming together due to a very positive, goal oriented team effort on the part of the VATUSA staff. I am so proud and thankful to be working alongside such a dedicated group of guys who have no fear about rolling up their sleeves and helping out.

more to follow.......

Brandon Bartell · « **Reply #25 on:** May 31, 2009, 06:42:45 PM »

Quote from: Andrew Podner

6:00pm Update:

Brandon, I just saw your post, thank you for the offer. I may contact you shortly.

Myself, Michael (USA3) and Rob (USA9) have been collaborating for the last few hours on the cert center and I am pleased to report that we have overcome the worst of the obstacles which was fixing the authentication system. At this time we are going to break for dinner and then come back to working on the cert center. Essentially what is left to be done is to update the form pages to start working right. This is a fix that we have already performed several times in other systems, and I feel like we will be able to overcome most of the problems with relative ease.

This is a very good development on two fronts, first that we are getting the system back online, and second, that this is coming together due to a very positive, goal oriented team effort on the part of the VATUSA staff. I am so proud and thankful to be working alongside such a dedicated group of guys who have no fear about rolling up their sleeves and helping out.

more to follow.......

I am going to PM you my number. Give me a call. I have done a lot of work with that PHPTest and I know how much of a pain it is to work with. I am happy to help out with anything that needs to be done.

Andrew Podner · « **Reply #26 on:** May 31, 2009, 09:43:49 PM »

********EDITED AT 9:50pm******* 9:30 PM UPDATE
The bad news:
-I'm Tired

The Good News:
-Cert Center is working in safari and IE7, and Firefox
-The instructor site / promotions panel is back online
-The ATM control panel is back online and functional as far as I can tell (if anyone accepts a transfer today let me know if it works.)
-All data and records are intact as far as I can tell
-The training center is still up and running
-Email is running
-New Member Application is working and the Basic Controller test is working
-Forums are working
-Everything that is broken can be fixed!

Cert Center:

It is possible to assign and complete exams in the cert center; however, the whole system is scheduled for replacement and we will commence work on this immediately.

Tests will not be added or edited until a new system is up and running. Staff members needing a change to a local test can contact me and we will work it out on the backend.

Thanks again for your patience. If anyone has problems, let me know.

Josh Guyer · « **Reply #27 on:** May 31, 2009, 09:49:32 PM »

You're doing an amazing job, Andrew. Keep it up.

Andrew Podner · « **Reply #28 on:** May 31, 2009, 10:16:00 PM »

I would like to first thank everyone for the support, patience, and understanding as we have worked to resolve these problems and challenges. The outpouring of wonderful comments is very flattering and humbling, and I appreciate it deeply.

I have described the last couple of days to the other VATUSA Staff members as putting band-aids on a gunshot wound, but I think we have bandaged this thing back up enough to get back to business while we finalize the VATUSA Strategic Plan for Information Management Systems over the next few days.

Some special thanks go to Rob and Michael for their time in helping me crack the code on the certcenter problem, their input was invaluable.

Thank you all again, and we will keep patching glitches as we find them

Tom Seeley · « **Reply #29 on:** June 01, 2009, 08:56:31 AM »

If you retired today (please don't) you would go down as one of the greatest data techs that VATUSA has ever had. You've done a masterful job in getting all this sorted out and working again, and over a weekend at that! Everyone on the system owes you a debt of gratitude.
[size=](Or at least a beer!)[/size]