Tuesday, January 12, 2016

Nimble 3 years later...

A long time ago, in a not so distant IT department I used to blog a lot.  Not sure what happened, but it's been way too long since my last post.

I still get questions about our Compellent to Nimble move and since it's been over 3 years I think it needs an update.

Q. Is Nimble really as good as they claim?
A. For us, Yes!  It works awesome, just set it and forget it.  There is no tweaking volumes or RAID levels or disk sets or anything.  For us, in an all VMWare iSCSI environment it works great.

Q. What kind of compression do you see with Nimble?
A. Right now we are seeing 25% compression on the primary volumes and 2X compression on the snapshots.

Q. Does it work well with Lotus Notes?
A. Yes, in fact our #1 volume for cache usage is our Domino volumes

Q. Are you seriously still on Lotus Notes
A. Yes, for now.

Q. Have you have any hardware issues?
A. Yes, we have had 1 Disk fail, several controllers and even had to replace the chassis because an LED wasn't working.

Q. Are you scared because of all the hardware failures?
A. It's not that many failures and No.  Everything is redundant and we have not had any downtime because of the Nimble.

Q. Are snapshots as awesome as they say?
A. Yes, yes they are.  We currently have over 4,000 snapshots and use this as our primary backup method.

Q. Does replication work efficiently?
A. Yes! This is the number 1 reason we choose Nimble.  Snapshots and replication is awesome.

Q. Does Nimble support SRM?
A. Yes it does, but I wouldn't recommend it.

Q. Why don't you like SRM?
A. For us, SRM was an unreliable, time sucking POS solution.

Q. Ok, what do you use instead of SRM?
A. We switched to Zerto and are loving it.  Switching from SRM to Zerto was like switching from Compellent to Nimble.  It just works and you don't have to think about it.

Q. After 3 years are you looking to replace it?
A. No, in fact we are looking at upgrading it.

Q. How easy it is to upgrade cache?
A. Not sure, but we will be doing so soon.

Q. How easy is it to add an expansion shelf?
A. Not sure, but we will be doing so soon.

Any other questions?  Post a comment and I will try to answer them.

SageTV 9, Windows 10 and OpenDCT

SageTV is an awesome HTPC (Home Theater PC) software package. I've been running it for over 7 years now and it works great for us. It has just recently been updated slightly, mostly bug fixes and getting it up to date.  Here is how to do a clean install on a new machine.

SageTV version 9, wait what, I thought SageTV was dead? Yes and no, since being sold to Google all official developement has stopped.  However Google has decided to opensource it and the awesome people in the SageTV community have decided to update it.

My latest server build had been running flawlessly for over 4 years, I then had some hard drive corruption and even though I had backups the system was never the same afterwards.  About 6 months ago I rebuilt it on different hardware and it's been a mess ever since. I have decided to rebuild my old server with the newest software I could.  Here is brief explanation of the steps required to get you up and running with SageTV v9, Windows 10 and an open source cable card tuner called OpenDCT.

Task 1.  Install and Update OS
  1. Install OS of choice (For me it was Windows 10 Pro 64bit)
  2. Update the OS
  3. With Windows 10 it's very easy and quick.  I am loving Win10 so far.
Task 2. Install Pre-Requirements
  1. Install MS C++ 2010 Runtime (required for java 7)
  2. Install Java 7 32bit - This is super important, even if you have a 64bit OS, install the 32bit version of java 1,7.  Not 1.8 and not 64 bit.  This is a current limitation for several pieces of the SageTV puzzle. Developers are working on getting it updated.
  3. Install LAV drivers - required because Windows 10 doesn't include any Codec's
  4. Install drives for your network tuners.  Either Silicon Dust or Ceton are supported.
Task 3. Install SageTV 7.1.9
  1. Install SageTV 7.1.9
  2. If you need Cable card support you have 3 options.
    1. SageDCT - I've used this for years with no problems until recently. It's my understanding that it needs some updates and the creator just doesn't have the time to commit to it.
    2. PrimeNetEncoder - For SiliconDust HD Prime's - I tried it and had issues getting it to work
    3. OpenDCT - the latest opensource cable card tuner support solutions.  Installed with no issues and after a few tweaks is working great for me.
  3. I would suggest starting the SageTV Service first
  4. Open SageTV and configure it, add your tuners and channel line ups.
Task 4. Tweak some settings.

1. Increase java heap to 1gb - Since Sage uses Java, it needs more ram allocated to it.
  • click start
  • click run
  • type regedit and click ok
  • expand SOFTWARE
  • expand Wow6432Node
  • expand Frey Technologies
  • under SageTV in right window doubleclick on JVMMaxHeapSizeMB
  • set as follows
  • 400 hexadecimal
2. Set your machine to reboot periodically. Due to memory leak issues, I set my machine to reboot every 3 days. Create a scheduled task to use shutdown.exe /r to reboot your machine.

Task 5. Test your setup
Verify you can tune channels and record shows

Task 6. Upgrade to SageTV v9
 1. See the SageTV 9 FAQ

I have been running this setup for several weeks now with no issues.  No corrupted recordings, no slow menus or anything.  Even though the future of SageTV is unknown, this system still works awesome for my family.

Monday, November 5, 2012

Windows Surface RT - mini review

Being heavily addicted to all things mobile I just had to go check out the new Microsoft Surface at the St Louis Microsoft Pop-up store. Well it's not really a store, here's a picture of it. Yep, that's it.

I have to admit after playing with it for about 10 minutes I really liked it and I almost bought one.  Now that being said I have been running Windows 8 on my main desktop and laptop since it hit RTM.  I have gotten use to it and now prefer it over Windows 7.

I was all set to buy one and start testing it out in our corporate environment. However there are a couple reasons why I didn't buy it. If I needed to test a Metro app I could just do that on my laptop or desktop, no need to purchase another piece of equipment just to test some apps.

The next reason was simply apps.  Our must have app is Lotus Notes. There is no Lotus Notes Metro app and I highly doubt there ever will be. I tried to connect the built in mail app to Lotus Traveler via activesync and it connects but does not actually work.  I asked IBM and they said it's not supported.  Why would it? if this worked you could get rid of the Lotus Notes client.  So I see it never being supported. We can always use the browser to get to Lotus Notes right?  Nope, iNotes isn't supported on IE10 yet. Plus the fact that with no client you take away one of the greatest features of Lotus Notes.  Offline access.

So let's move on.  We don't have our number one productivity app, what else can we do with RT? Ok, let's look for a VPN client so we can use the number one selling feature of RT, Office. No Sonicwall client, no Cisco client.  I asked the MS store employee and he said "Advanced security apps like VPN will be supported on Surface Pro, due out in January."  So that kind of tells me VPN will never work on RT.  So let's move file storage to the cloud.  Ok, we use Sharefile for that. Again, no Sharefile app.  You see where this is going.

Does anyone know if Windows RT would support MS Direct Access if we went and set that up? I've read that it is much easy to implement with Server 2012 and Windows 8. Not sure which version of Windows 8 though.

Ah, I have my ace in the hole that always works.  Citrix.  Yes! There is a Citrix app in the store.  I download it and can't connect.  That's it i'm done and leave the store without buying a Surface RT.

Upon further research it looks like the Windows RT Citrix Receiver uses a newer Citrix Infrastructure that we don't have in place.  It fact it requires a paid upgrade on some Citrix licenses to get to the new infrastructure.  Windows RT is dead in the enterprise if you ask me.

A final note on the Microsoft store, the employees were very friendly and offered to help several times.  I asked several questions to different employees and you can tell some were technical and some were just there for the job.  I also witnessed two Surface's being sold and one being returned.  I asked how many they have sold and was told about 300 on launch day and about 425 total for the first week.

Friday, September 28, 2012

SRM 5 works! What a letdown.

This week we successfully tested SRM 5 and guess what? It worked.  We failed over, everything worked without a problem.  We failed back and guess what?  It worked again with no problems.  Not just the SRM "Test" we did a full-on "fail over" and "fail back."

This is all very anti-climatic for me. Our road to a working DR site has been a very rough, long, stressful, frustrating experience. We dealt with so many problems along the way; staff cuts, SRA problems, professional services problems, replication problems, failed WAN upgrades, etc... I should be excited, I should be jumping for joy.  A huge weight has been lifted off my shoulders. It's not fully configured yet, but the fact the we tested with no problems is a huge step in the right direction. However, i'm not super excited. I'm more aggravated that it took so long to get to this point.  Maybe after we have it fully configured and run production out of the DR site for a day or two will I be excited.

So what's the key to this working with no problems?  Nimble Storage. Yep, I replaced my storage to get all this to work without a problem.  It's all because how they perform snapshots and therefore replication. With our previous vendor we were 1-2 days behind on critical data.  Now with the Nimble we are 1-2 HOURS behind on critical data and 1 days behind on all data.  Yes, we are replicating our entire infrastructure daily.  All over the same 10mb MPLS pipe which is limited to 3mb during the day for replication traffic.

I would highly suggest you look at Nimble even if your not looking for a SAN replacement.  I wasn't looking to replace my SAN, in fact I was pretty dedicated to Compellent before my latest round of problems. Nimble does everything they say it does, it does it well and at a cost that is about the same as keeping my Compellent over the next 3 years.

If you are looking for a good deal on some Compellent gear let me know.  I have three Series 30 controllers,  1 shelf of Sata, 1 shelf of fiber and 2 shelves of SAS.

Wednesday, September 12, 2012

VMWare maximum VMDK file size with snapshots

Here is a problem we recently ran into with our new Nimble install.

We did our research, we asked the questions, we looked up VMWare KB articles and Nimble best practices guide.  Everything we read said the maximum file size is 2TB minus 512bytes.  Even VMWare support told us this.

This is all true, UNLESS you want to snapshot the machine that has that VMDK associated to it.

Then the maximum VMDK file size is 1.9845TB. To be safe we now make VMDK's 1.98tb.

This was very very frustrating for us because the error message VMware was throwing was not descriptive at all.  We were getting this.

File is larger than the maximum size supported by datastore '

What was also extremely frustrating is that we called Vmware several times on this issue and no one really knew the answer until is was escalated up the chain. We were told repeatedly that we should be able to take a snapshot on a 1.99tb vmdk.

Here is the Vmware KB article they sent us explaining the problem.  

Things like this really make me question using VMWare and paying for premium support with them.  In my opinion this should have been resolved in less than 30 minutes, not 6 hours.

Once the Nimble install is done, I'll start looking at Hyper-V in 2012 and see what it offers.  XenServer has too many problems to replace VMWare in my environment.

Tuesday, September 4, 2012

Compellent to Nimble Day 1 – Install and Update

Today was kind of a anti-climatic.  The Nimble’s were supposed to arrive by 10:30, they didn’t arrive until 3pm.
We waited all day for them to arrive, once they did it was go time.
In 1 hour we had both array’s unboxed, racked, stacked, cabled and powered up.
Two Nimble CS240’s and a Dual Series 30 Compellent with 2 shelves of disks.
The cabling looks terrible, but that’s temporary for the DR array to seed data.
Within 2 hours of these array’s arriving they we racked with an initial config and upgraded to version 1.4
Tomorrow we will get into the VMWare setup, create some volumes, protection policies, replication schedules, etc…

Monday, September 3, 2012

Citrix XenServer 6, iSCSI and Compellent – Part 2

It’s safe to say after our last experience with Co-Pilot I was pretty disappointed, then I got my quote for 1 year of support, the first time we are paying for support since we purchased them, and I was shocked.  I wasn’t going to pay that much money for that kind of support.  I have been told over and over again that this is not typical for Co-Pilot, however it is typical of my experience with Co-Pilot.  If I get around to it I will blog about my experience with Compellent and SRM 5 last December. Also around the same time that I got the quote for support one of my friends was talking about a new SAN he bought called Nimble.  It sounded interesting and the price was amazing compared to Compellent. Seriously I starting looking at the Nimble just because of the price.

I decided to start looking at other options to see if it was even possible to replace the Compellent.  I never ever thought I would replace the Compellent, even though I was very disappointed with them. I started putting out some feelers on LinkedIn to see if anyone would buy my systems and to see what my support options were. Luckily I found a couple of business partners that are interested in purchasing all my gear and I got a lot of good feedback about my support options.

I even had a regional Compellent sales rep ask why I was considering not renewing support.  He asked if I was having problems with the product or with support. He also asked if my local Compellent team was aware of the problems. I replied with “Yes, I have problems with the product and with support.”  “Yes, my business partner is aware of the problems and so is my local Compellent team and they have done nothing to resolve the problems.”  Well apparently that was the quote that set Compellent off.
Within days of that appearing on LinkedIn I got a call from Compellent Co-Pilot management asking about our issues and they were determined to resolve the issues. Just a little more history on this ticket.  This case was opened on on June 18th, the day we converted to virtual ports was July 21st. So a month between opening a case where servers were locking up and replications failing until they suggested and we could implement a resolution. After the 21st I sent an email to my business partner and local Compellent team. No calls, no contact, no email, nothing.  Then after the LinkedIn postings I got a call on August 2nd. It’s now September 2nd.

Compellent has been all over us since that day.  They have assigned a tier 3 tech to our case, they call us almost every other with suggestions and to check on our progress.  However, they still can’t solve the main problem.  XenServer 6, iSCSI and multi-pathing.

The main problem is this. Compellent says multi-pathing with Xenserver 6 and iSCSI HBA’s works.  However, we can’t get the Qlogic 4062C iSCSI HBA to display both ports in XenServer 6. It works fine in 5.6, just not 6.0.  So we called our Citrix business partner and explain the problem.  They tell us that it is not supported and we are wasting time trying to troubleshoot it.  We tell this to Compellent and they insist it works and to open a ticket with Citrix. We call our Citrix business partner again and have them open a ticket with Citrix.  They call us back and say that card is not supported in XenServer 6, so they can’t help us.  It turns out, NO ISCSI HBA’s ARE CERTIFIED ON XENSERVER 6.0.

In the mean time we had been investigating the Nimble array quite heavily.  The price is right and it all sounds great and looks awesome on paper. Compellent wasn’t making any progress at the time and the Nimble sounds like it will solve our replication issues and improve performance.  Over the next 3 years buying two new Nimble’s was about the same price as paying Compellent support and upgrading my series 30 controllers. We decided to participate in a Nimble proof of concept and it starts next week.  If it works, they will replace the Compellent.  This was all decided and signed before we found the real problem, no iSCSI HBA’s supported on XenServer 6.0.  Since virtual ports doesn’t work with software iSCSI, we are kind of screwed.

Since this problem is with XenServer and not Compellent we have made some decisions.
  1. We are no longer trying to get multi-pathing to work in XenServer
  2. We have decided to flatten our iSCSI network to one subnet on two switches
  3. We have decided the long term solution is to replace XenServer with more Vmware licenses
I plan on blogging about the Nimble install and data conversion as well.  So stay tuned for more posts on that.