Director's Blog - Tier II Web Servers

One step forward...two steps back

Tue, 19 May 2009 17:30:00 -0800

Well, we ran into yet another issue with the Cisco ACE appliances. In order to effectively troubleshoot the issue, we are rolling back the DNS change we made yesterday. I apologize (again) for the delay on this. If we can't get the Cisco ACE to work properly, we'll update the Cisco CSS to use the new web servers. That way, you can still get the benefit of the hardware and software upgrade. I'll post more information tomorrow after we conduct more testing.

Launch!

Mon, 18 May 2009 17:20:00 -0800

Well, we updated the DNS records today for the 'www' and the 'ozone' sites, but only for internal users. Client browsers outside of the OHSU network are still using the older servers. (We'll update external DNS in a week or so.) We've run into a few configuration issues that were missed, but we've also been resolving them as they are reported. If you are experiencing any problems with the site, please contact the ITG Helpdesk or Web Strategies. Thanks for your patience!

New Launch Date

Fri, 15 May 2009 11:24:00 -0800

We're going to launch on Monday, May 18th at 9:30am. Hopefully, this should be a seamless transition this time. I've reviewed the old CSS rules line by line (all 5186 of them). We're very confident that the new servers are now configured to support every URL that the old servers were supporting. Wish us luck! And of course, be ready to test your websites Monday morning!

Postponed.......yet.....again

Thu, 30 Apr 2009 11:07:00 -0800

I just discovered that the IT engineers who can update the DNS servers (needed to launch Tier II and revert back if there are major problems) will all be busy today taking examinations. Therefore, we are re-scheduling the launch for Monday, May 4th, at 11am.

Tier II Web Upgrade

Tue, 28 Apr 2009 16:40:00 -0800

We are planning to actually launch the upgraded servers on Thursday, April 30th at 10:00am. I really, really, really hope I don't have to retract this announcement! Damon

Tier II Web Upgrade

Tue, 21 Apr 2009 09:25:00 -0800

The Network Engineering team will be applying a firmware patch to the Cisco ACE network appliance modules on Thursday, April 23rd. Web Engineering would like to wait a few days after this patch to ensure that the ACE modules are stable after the upgrade before we make the DNS change that send traffic for Tier II web sites to the ACE. Thanks for your patience!

Tier II Upgrade - More delays

Wed, 15 Apr 2009 09:58:00 -0800

I'm sorry to have to report that the Tier II upgrade has once again been postponed. I probably jumped the gun a little on announcing that we would launch yesterday. The issue at hand is that the Cisco ACE (load balancer) is having some problems. Specifically, one of our VIPs (Virtual IP addresses) is failing, and we don't know why. The failing VIP happens to be the one that serves http://dev.ozone.ohsu.edu. The 'production' VIPs seem to be working. However, until we identify the root cause of this issue, I'm worried that the same issue might affect a production VIP. Therefore, we are once again postponing the launch. I will keep this blog updated with status reports as often as I can. Thanks in advance for your patience!

Tier II Upgrade - Today!

Tue, 14 Apr 2009 10:30:00 -0800

Ok all. We are finally ready to launch the new upgraded Tier II servers. At 11am today (or shortly thereafter), the DNS records for www.ohsu.edu and ozone.ohsu.edu will be updated for INTERNAL network clients only. Anyone on the internal network browsing the OHSU or O-Zone sites will be using the new servers. All external web browsers will continue to view our site on the old servers. If you experience any issues with the new servers, please follow our standard support notification process by completing our support form at: https://www.ohsu.edu/webstrategies/support.cfm. Thanks for your patience!

Tier II Web Infrastructure Upgrade - Round II

Mon, 13 Apr 2009 10:03:00 -0800

UPDATE: Yet another postponement. I apologize for the series of delays here. This time, the network engineering team has discovered a bug with the Cisco ACE appliances. They are working with the vendor to get the issue resolved. Currently, they are expecting to have the ACE issue fixed later this week. I'll keep this blog updated with status as soon as we see a change. For now, Tier II is still running CFMX 6 on the old 'cluster'.

Tier II Web Infrastructure Upgrade - Postponed

Fri, 03 Apr 2009 09:34:00 -0800

Hi all. We discovered an issue with the NFS Mounts and symbolic links that would have affected lots of applications in the production environment. We should be resolving that today and will begin testing as soon as possible. At the moment, the revised ETA for Tier II launch is Tuesday, April 7th.

Tier II Web Infrastructure Upgrade

Thu, 26 Mar 2009 17:40:00 -0800

If your content is on the Tier I environment (CommonSpot), this upgrade will not affect your site. This upgrade is only for web content on the Tier II environment. On Tuesday, April 2nd, computers on the OHSU network will begin using the upgraded Tier II web hardware for 'www.ohsu.edu' and 'ozone.ohsu.edu'. External visitors to these sites will continue to use the existing, original hardware. A follow up DNS change on April 13th will be implemented to allow external visitors to the OHSU site to use the new hardware as well. You can test your site on the upgraded Tier II environment at one of the following URLs: - http://test-www.ohsu.edu - http://test-ozone.ohsu.edu Please contact Damon Gentry (x4-2601) if you have any questions.

New Testing Schedule

Fri, 09 Jan 2009 10:29:00 -0800

In order to promote testing of the upgraded Tier II Development environment, we will be shutting down the old Tier II Development environment. On Friday, January 16th, the server 'SUNWEB1' will be modified to display a message to all site visitors. The message will state that server SUNWEB1 is no longer available, and testing should be done on the new servers. Here is a quick reference to map the old URLs to the new URLs. http://sunweb1 -- http://dev.ohsu.edu
https://sunweb1 -- https://dev.ohsu.edu
http://ozone-dev -- http://dev.ozone.ohsu.edu
The new servers are loading the exact same content as the old servers. Therefore, developers will still continue to put their site code in the same location as they have been in the past. Again, if you have any questions, please feel free to email us at: webadmin@ohsu.edu.

ColdFusion 8 - ready for testing

Thu, 18 Dec 2008 10:08:00 -0800

I'd like to announce that ColdFusion 8 is ready for testing on the following URLs: http://dev.ohsu.edu https://dev.ohsu.edu http://dev.ozone.ohsu.edu We will be preparing the new production servers to use ColdFusion 8 over the next few weeks. By January of 2009, we will be ready to conduct comparison testing of the old cluster and the new Tier II web environment. Here is a quick summary of our current web environments: DEVELOPMENT ENVIRONMENT You can continue to use SUNWEB1 as a testing server. SUNWEB1 is a ColdFusion 6 server. The new dev environment can be found at the URL listed above that start with 'dev.'. This environment is running ColdFusion 8. PRODUCTION ENVIRONMENT The existing web cluster is still our current production environment running ColdFusion 6. The new production environment will be ready for testing on January 5th. Questions? Please comment to this blog post or email Damon Gentry (gentryd@ohsu.edu).

Tier II Upgrade Status

Mon, 08 Dec 2008 13:13:00 -0800

Hello all, We ran into a slight 'snag' a few weeks ago and had to re-build the new Development environment. The issue had to do with how the NFS servers were offering up mount points. It turns out that Apache needs to mount the NFS file systems slightly differently than the ColdFusion servers. This has now been resolved and you can continue testing the development environment at the following URLs: http://dev.ohsu.edu https://dev.ohsu.edu http://dev.ozone.ohsu.edu We will continue testing the newly reconfigured Development environment for 2 more weeks. By the Christmas break, we should have the new Production servers ready for testing. We will be shutting down the SUNWEB1 web server on December 12th, so please be sure to test your code on the new hardware.

Tier II Web Development Testing

Mon, 03 Nov 2008 11:15:00 -0800

Web Engineering is happy to announce that the development environment for the Tier II (web cluster) infrastructure is ready for testing. The old development server (http://sunweb1) is still available and will continue to be available for the next few weeks. We are leaving this server up in order for you to compare how your code executes on the new server, as compared to the old server. The new development server is available through the following DNS hostnames: http://dev.ohsu.edu (for ohsu.edu sites)
https://dev.ohsu.edu (for secure ohsu.edu sites)
http://dev.ozone.ohsu.edu (for ozone sites)
There is an attached PowerPoint Presentation for you to view that describes the testing process in detail. Please click on the 'download' link attached to this blog posting. You can use the following url to report problems with the new server. While the Web Engineering team will be testing various sites on this new environment, we cannot test it all. We need your help testing the new hardware. Without your testing and feedback, we can only assume that the new environment will be ready for production use by December. https://www.ohsu.edu/webstrategies/support.cfm Finally, you can contact Web Engineering if you have any other questions: Damon Gentry
gentryd@ohsu.edu
x4.2601