SSL + Migration Plan Update - 09/05/2014

Importance: IMPORTANT
Author: albert
Date: Friday, September 5th, 2014 at 5:30:16 PM PDT
Blog post: http://withg.org/blog/index.php?post=1409963416

SSL + Migration Plan Update
======================================

Migration Delays
----------------
We are working on trying to find a new provider to consolidate our
servers to. One of the providers that we approached had a great price
of around $15/month. However, they have suddenly changed their policy
to not allow hosting a IRC server, which we wish to continue to provide
to the community as part of the OmniNET network.

As an alternative, we are thinking about moving to a different
provider. The alternative provider has been in the industry for a long
time (much longer than the $15/mo provider), and offers a similar
configuration for around $20/month.

What do you think? Would you be willing to contribute to that? (For
reference, the old server costs would total around $36.10/month.)

Let us know! Go here:

http://ourl.ca/21452

SSL Certificate Update
----------------------
Many companies and developers out there are announcing plans to end
SHA-1 SSL certificate support, including Microsoft, Symantec, and
most recently, Google. The new recommended standard is SHA-256.

That said, we decided to take the initiative and upgrade our SSL
security from SHA1 to SHA-256. Both HTTPS and ZNC have been upgraded to
SHA-256 signatures.

What changes? Nothing much - just better security, that's all! If you
look at your browser's security page when on https://withg.org/, you may
notice that a SHA-256 section now exists!

With ZNC, you may also notice this change with Hexchat/XChat, with the
following message:

 Public key algorithm: rsaEncryption (2048 bits)
 Sign algorithm sha256WithRSAEncryption

The important part is the "sha256WithRSAEncryption", indicating our new
shiny SSL certificate! For reference, this was the old one:

  Public key algorithm: rsaEncryption (2048 bits)
  Sign algorithm sha1WithRSAEncryption

Reeks of bad security, eh?

That's it!
----------
Make sure to respond to our proposal above!
http://ourl.ca/21452

Let us know if anything breaks, or just comments/suggestions in general!

Okay, I think I know how. Here's my guess.

Your file consists of "a();" and you compile it with gcc cheat.c -o cheat -D'a()=insert whole code here'.

Correct! Almost!

This is what it looks like:
a
b
all:
        gcc test-noincl-crazy.c \
           -include "stdio.h" \
           -D'f=for(i=0;i<4;i++){' \
           -D'a=char s[255],*p,r[4],i=0,x=0,c[]="code";' \
           -D'b=void main(){fgets(s,255,stdin);f p=strchr(s+x,c[i]);if(!p){printf("0\n");return;}x=p-s;r[i]=x+1;}f printf("%i\n",r[i]);}}' \
           -o test

To build: Place the two files in the same directory, and run make -f Makefile.crazy. Requires GCC and make.

Again, this was NOT a valid entry, and was just submitted for fun and games!
But if there's any lesson from this, it's that C macros are very, very powerful!

Requirements to be a team member should be Requirements to efficiently work on this project

Changed it to "Requirements to being a GREAT team member" and added this little blurb:
Note that you can join regardless of experience, but knowing the above allows you to contribute the most to the project.

Is knowing how to go to a folder, list all files in a folder, and running and building programs seen as enough CLI knowledge?

Pretty much! Really, it's just the ability to find/get CLI help (cmd /? or cmd --help), and from the help, run the necessary tools in CLI. The tool in question is our build tool, Premake. A quick start guide for it can be found here.

Anyway, it looks like I have a lot to learn to be able to start on it, but it looks like a great challenge, and I'd love wxWabbitemu to be more up to date - m.

Though can I ask you, alberthrocks, to then please either join our team or at least look at our GitHub regularly to give advice and feedback? This project is quite different from other projects I've done in the past, and I think this is the case with more people from the team.

Awesome!

And definitely - the whole goal is to mentor those working on the code so that it isn't too much of an undertaking! (I might be busy, but at the very least, I'd like to help get things going, and try to answer any questions people may have.) The only thing is that the final code will be at our repo, alberthrocks/wxwabbitemu, for QA and centralization of efforts. (No worries, we'll give credit where credit is due!)

BTW: my main OS is Linux (Ubuntu), but I have Windows 7 on a virtual machine, so I can test on both.
Oh, and I have programmed in ASM in the past, so I know the basics of how the hardware works. I'd need documentation for the details though.

I'm dual booting Windows 7 and Linux Mint (LM is the OS I tend to use more).

To be honest, I don't have much hardware knowledge, so I tend to trust BuckeyeDude's implementation and work to integrate his code from there. However, knowing the HW is a big plus and can help when debugging wxWabbitemu and adding the Wabbitemu debugger later!

poof

True although not to be rude, in the past two years there were several messages that made it look like Withgusto was in constant danger of closing if it did not meet a certain donation goal. It might be best to avoid "Boy who cried wolf" scenarios as people will start taking the service for granted while donating less, thinking it's not needed after all.  (Not to mention if people discover you are 18+ they might be less tempted to donate as they'll assume you have a job to pay the server)

But it was in danger of being closed. Albert is a poor student. He doesn't have a steady income like I do. Omni is never going to go away while I'm still making money. I do wish I didn't have to pay for it all, but not enough people to monthly donations for me to reimburse myself yet.

It's not a boy who cried wolf situation, it's a "Money is getting really tight here, and I need help to keep paying for the server, otherwise I might have to eat less this month" situation.

Alright, time for some clarification...

It is true that I do like to joke around about shutting down the server, and with that, ask for donations as well. But those are not threats in any way, nor are they meant to be. Yes, there are server owners out there who do make serious threats... but I'm not that kind of person!

Server downtimes are not caused by me - they are usually downtime on my VPS provider's end. Occasionally, if it is my fault, I forget to pay the bill. (This happens during school, when I'm really busy...) In any case, there will NEVER be a case where I just stop paying - at least not without notifying everyone about it!

In terms of finances, my money is always in flux. I'm not exactly well off, but I'm not exactly 100% poor either. That's because sometimes, I have enough money to fund the server. Other times, I might be in a financial pinch.

This instability of finances is what led me to create the donation page, withg.org/donate. (In fact, if I had a stable income, I would probably never have created that page!) The money that goes through there does, in fact, help me a LOT in making sure that the server doesn't fall through. I can probably name one time where the donation money helped when my money really dipped...

And then there's the migration. The migration is meant to do two things: 1) Reduce server costs to $15/month or lower; 2) Gradually ease off my server payments to make it completely 100% community funded. The reason it failed last time is mainly that I was busy, and that it depended on pretty much the tool working. The other reason was probably that the amount of money I had was enough to keep things going, at least for a while. So, there was that.

Now we're here: migration #2. The motivation is mostly the same... except for one more thing - my finances.

Yes, you're right that I do have a job. No specifics will be mentioned, but I did get plenty of money from that job. But it was just an internship - and that money (and really, my bank account) will get drained very soon - as soon as TOMORROW. Yup - all of my money is being drained into my college tuition.

Of course, if you have a good memory, you'll probably ask, "Why didn't he have to drain his bank account last year?" Again, I won't go into specifics, but stuff happened (TM), and now I'm in a position where a good chunk of my money will have to go to tuition. (That's why, this time, I made sure that the migration was much more thought out.) If you really want to know the specifics, please PM me and I'd be happy to show you everything - my bank account, my bill, everything.

Finally, the last plan... withgusto shutdown. I understand your worries. As I mentioned before, I joke around about it a lot... maybe a little too much. (April Fools was a particularly noisy time...) But no doubt, I think and worry about it a lot! And here's what I have to say about a shutdown: if, in any case, there's a need to shut withgusto down due to finances (or anything, really), I will warn everybody in advance and make sure backups are made before everything goes down. In fact, I'll make a personal backup of the ENTIRE server just so that if anybody didn't get to backup in time, at the very least, I would have a copy to give them. I don't want to leave anybody in the dark and have my server disappear overnight - again, this is a community server, and I feel obligated to make it the best it can be, even while it is sinking. (Which hopefully will never happen, or at the very least, won't happen for times to come.)

Crazy stuff aside, let's hope for the best with the migration!

Just noticed, why isn't this on http://withg.org/blog/ ?

Good point - done! http://withg.org/blog/index.php?post=1409006396

Just updated the first post with the survey! Go ahead and take it!

https://docs.google.com/forms/d/1tDwSGjn_qul5_TsffkAdVn5SwZspgTGORTeVOkW3sT8/viewform

We're just about to hit Stage I, stay tuned!

Migration Plan
You heard it right.

We’re finally moving! Unlike last time, we plan to actually get it done this time... hopefully.
More importantly, the methodology for migrating from our old servers to a shiny new one will be different. Really different.

Old versus New (or: how not to fail at migration again)

Unlike last time, we will NOT be using an automated tool to migrate everything. (No worries, if you’re looking for that magical tool, it’s still in development, but it’s easier to test it on a physical computer than a virtual server...)
In addition, we will not backup as much data as we did last time. This time, we will hand pick directories that the withgusto administrators deem to be core to withgusto services. The good news? We’ll make sure your data gets to the new server safely. Bad news? If you’re a weirdo (as in someone who doesn’t store all of their data in their home directory, /var/www/username, or in an active core service offered by withgusto), you may be out of luck.
What does it mean? It doesn’t mean we’re tossing your data in the trash - I think pitchforks might come haunt us if we did that - but it does mean that you are 100% responsible for backing up (and moving it) yourself.
We will give you plenty of time to do the move - but you must do it yourself.

The sky is falling! AHHH!

No worries, we have one other thing. If you are a weirdo, BUT your data is consolidated and you are able to set things up by yourself, you can tell us which directories you want us to move, and we’ll happily move them for you, gratuit. No hidden fees. The only condition is that the files MUST be simple to move. If it’s a tangled web of symlinks, or scattered files throughout the file system... we’ll probably have to ask you to move that. (It’s for your own good - if we forget to move complicated files, or something goes wrong, it’s going to go bad for both of us! We want to make sure that everything makes it to the new server safely!)

OK, what now?

The migration, like last time, will take place in stages. Of course, we’ve learned our lessons from last time, so the stages will be very different.

STAGE I: Announcement of Migration Files and Folders + Backup Survey

At this point, we announce the files and folders that we will move. At this time, we will ask withgusto users to help out by compressing files and moving large files off the server so that we can prepare to backup everything. We didn’t forget about backup2, either - we will also announce files and folders for that as well.
At the same time, we will also open a survey for files and folders to move for the oddballs out there. This survey will be used to determine additional files and folders to backup.
We will also make sure we truly back up everything - if we forgot anything, please let us know!
The new server will also be up on new.withg.org for those who would like to do initial migration stuff.
Finally, we’ll announce dates for STAGE II and STAGE III - at STAGE II, the survey will close.

STAGE II: User Directory Lock + Initial Migration Starts

At this point, withgusto users will no longer be able to login, and the migration process will begin. The initial migration process will simply be moving files to the new server, re-creating the users, and setting the appropriate permissions.
At this time, we will have finalized the list of important withgusto files and directories to back up, and we’ll list them in our post for STAGE II. The survey will be closed, and the files and directories that we’ve approved to backup will be listed as well.

STAGE III: User Directory Unlock + Service Migration Starts

At this point, users will be able to login again to perform manual migration. At the same time, withg.org will be renamed to old.withg.org, and the new withg server will be renamed to withg.org. Why? Service migration, of course!
We hope to have most of the services up and running on the new server. Of course, we don’t like downtime as much as the next person, so we’ll keep the old server’s services running, and gradually have people migrate over to the new server via simple DNS propagation.
This stage will last for around 2 weeks.

STAGE IV: Old Server Termination Warning + Old Service Termination

At this time, we will post a warning for the old server’s termination. We will also terminate the old services in favor of running solely on the new server. We will merge any data, if necessary, and then shut down the old withgusto services for good!
At this time, any files that need to be backed up should be backed up now!
This stage will last for around a week.

STAGE V: Old Server Termination

For the last and final stage, we will announce a 3-day grace period for shutting down the server. Once that grace period is over, we’ll shut it down... and wait for another 4 days for any last-minute requests! After a combined total of a week, the old server will finally be terminated.
Like the previous stage, this stage will last for around a week.

And finally...

STAGE 0: Announcement and Preparation!

We’re at this stage - announcing our intent and allowing everyone to prepare! One other thing we didn’t mention... if everyone on withgusto agrees to move onto the next stage, we can make the transition MUCH faster! We can do this with the following:

• Notify withgusto users you know about the migration!
• Help us out by backing up your larger files and directories, and removing them from the server! (No worries, if you need a secondary backup, you can upload them back on the new server once it’s up!)
• Check and make sure you aren’t “weird”. If you have files elsewhere, make a note of it! Try to backup it beforehand - if you can’t (or don’t want to), write down where it is and let us know in the survey!

Final Total Time: around 6 weeks... or less!

...and that’s pretty much it! Let us know if you have any questions, comments, or concerns!

Survey
https://docs.google.com/forms/d/1tDwSGjn_qul5_TsffkAdVn5SwZspgTGORTeVOkW3sT8/viewform