There are two super new features for BigV available from today.

Firstly we’re now offering archive-grade storage on BigV: that’s 50GiB of lower-performance direct-attached disc space for £2 per month. That should be ideal for backups, logs and other data where performance isn’t critical.

This complements our standard sata storage (for system discs) as well as the sas and ssd grades which are based on faster storage technologies.

If you run a single VM, your archive-grade storage is guaranteed to be on a separate storage pool from your main disc. You can add 50GB to my-machine (for example) by typing:

bigv disc new my-machine archive 50

To start using archive storage, you’ll need to update your BigV client to 0.8.0 which is out today, and fixes a few other issues.

Secondly we’ve put out a first version of the BigV web manager at https://manager.bigv.io/  You can use this to create, delete and upgrade virtual machines, as well as viewing their consoles without installing any special software (thanks to the excellent NoVNC project).

It has a few rough edges (especially group and V-Key support), and the interface will undergo some revisions over the next month, but it’s already very useful if you’d rather manage your hosting infrastructure from the web than the command line.

Finally, if you’re a BigV power user and want to keep up with the development of the UK’s most ambitious new public hosting platform (scientifically proven), you might want to join our bigv-friends private mailing list. Just email bigv-friends-join@lists.bytemark.co.uk. You can also sometimes get heads-up on new features through our @bytemark feed on Twitter.

Comment...

Here at Bytemark, we like Free and open-source software. And one of our favourite projects is QEMU. QEMU emulates a bundle of common server hardware – including a CPU, discs, memory, a network card, etc – and runs your operating system on it. It’s pretty much the code that runs every “cloud” or virtual hosting provider in the world.

Of course, servers can contain lots of different pieces of hardware (many of which can be emulated by QEMU), and different operating systems have varying levels of support for them. When deciding how to configure QEMU for our BigV virtual machines, we wanted the defaults to be as fast as possible, but we also wanted to be able to support a wide variety of operating systems.

We met these contradictory goals by introducing the idea of ‘hardware profiles’. Until today your virtual machine was given a profile called virtio2011 (or, if you asked for it, compatibility2011). By picking one hardware profile or another, you were deciding what hardware QEMU would emulate for your VM - fast-but-narrowly-supported, or slower-but-widely-supported. The hardware profile also decides which version of QEMU is used to run your virtual machine, with the 2011 profiles using a modified version of QEMU 0.15 – which actually dates from 2008!

Of course there have been a lot of changes to QEMU in 5 years. While we’re happy with the hardware that QEMU presents, and its stability there are still loads of bug fixes, performance enhancements, and behind-the-scenes features we’d like to start using in BigV.

As of right now, any new virtual machines you create on BigV will use the new virtio2013 hardware profile, which uses the brand new QEMU 1.5.0. If you didn’t specify a hardware profile when creating a virtual machine, it will have defaulted to virtio2011, and next time it’s stopped (by running bigv vm shutdown or bigv vm restart, for instance), it will be automatically upgraded to virtio2013. This is almost invisible to your guest – when it checks the CPU’s model name, it will see “QEMU Virtual CPU Version 1.5.0″ instead of “0.15.0″, but that should be all.

VMs that were created with a hardware profile of compatibility2011 will, by constrast, not be upgraded to compatibility2013 just in case there’s a very subtle change that somehow breaks your OS. We also won’t automatically upgrade a VM if you’d specified virtio2011 explicitly, just in case you had a reason to do so, and we’ll stick to that for any future VMs where you choose a profile.

If you’d previously specified compatibility2011 or virtio2011 for a VM and you’d like it to be upgraded to the 2013 release, the newest version of the BigV client (0.7.3) includes support for setting, locking and unlocking hardware profiles, as long as the machine is turned off. Do pop in a support request to support@bigv.io if you have any problems. We’d also like to hear from you if there’s new virtual hardware you’d like to see added to BigV. For now we think the virtio and compatibility profiles cover most use cases but please tell us if you think we’re missing a trick.

Finally, we recently identified a bug in QEMU that was harming disc I/O speeds. We’ve fixed this for all our hardware profiles. This reduces I/O request latency by a factor of 40-200. This translates to some fairly drastic improvements in the performance of everyday tasks – compiling a typical C++ application was seven times faster, in one example – and differences between the various disc storage grades should now be more obvious. So far feedback has been positive, but if you’ve had I/O problems now is the time to reboot your VM, check again and let us know.

Comment...

Last week, we ran a competition to find the most exciting thing on BigV. I’m pleased to announce we have a winner!

With Associates have built the site for Code Club, “a nationwide network of free volunteer-led after-school coding clubs for children aged 9-11″ and host it on BigV.

We love the idea of giving schoolchildren a fun after-school activity which can easily turn into a lifelong interest and marketable skill. Most of us at Bytemark got started that way, but experience of computers in the 80s and 90s came from generous parents, uncommonly ambitious schools and teachers going above and beyond the call of duty. We like that Code Club is helping to level the field.

Jamie White of With Associates:

Code Club came to us at the start of the year to see if we might be interested in building the next version of their website. We jumped at the opportunity, and the past few months have seen us working closely with co-founder Clare Sutcliffe to realise her vision. It launched last month, and so far it’s been a great success.

The Code Club site (pictured above) uses a pretty recent set of technology:

• Slices, With Associates’ in-house content management system (built on Rails);
• The database is MongoDB;
• It uses queues extensively, via MongoDB and the Qu gem;
• It plugs into a bunch of other services, including:
• Google for geocoding (soon to be switching to Postcode Anywhere)
• Mandrill for email delivery
• Intercom for CRM/support

Jamie will be at the Scottish Ruby Conference next week courtesy of Bytemark, and we look forward to seeing more children programming from theirs, and Code Club’s efforts.

Comment...

The highly popular ScotRuby conference is rapidly approaching on the 12th and 13th of May – a two day residential gathering of the Ruby community at the Crieff Hydro hotel in Perthshire with an exciting schedule of talks.

We’re giving away an all-expenses paid trip to ScotRuby 2013 to one lucky person and a friend.

Do you have the “most interesting use of BigV”?

Are you running a PBX/VPN service, perhaps you’ve written an exciting plugin for your CMS, or perhaps something else? It doesn’t need to be related to Ruby or be a website but it does have to be hosted on BigV.

If you’re not on BigV yet, whether you’re a new or an existing customer, getting online is simple – there’s still time to enter.

One winner will get:

• Travel & food expenses
• 2 Conference Tickets
• 2 single occupancy hotel rooms Crieff Hydro from the day before the conference, until the day after (11th-14th)

And of course, we’ll blog about the winners

To enter, simply email competitions at bytemark.co.uk before 15:00 BST Friday 03/05/2013 with details of the thing you’re entering.

Unfortunately sponsored customers and Bytemark staff are ineligible for this competition – sorry guys, we love you already!

Winners will be notified by email & announced on this blog at the start of the week beginning 6th of May. There is no cash alternative. The judge’s decision is final. We might decide to award two individual winners if the first person doesn’t have a friend they want to bring.

The Crieff Hydro hotel where the conference will be held:

Comment...

It’s been a while, but we pressed the release button on 0.7.0 back on Tuesday – then quickly pressed the release button on 0.7.1 the day after! We’ve been somewhat distracted by under-the-hood improvements on the server side until recently – but that’s another story.

Eagle-eyed Linux users may have noticed the (very quiet) introduction of a Qt-based front-end in the last release; some work has gone into polishing this for 0.7, and we think it’s good enough for casual use now. Equally eagle-eyed Mac users will have noticed that a DMG-style installer has also been introduced for BigV, replacing the previous manual installer. Unfortunately, we couldn’t get the GUI client packaged in this form for 0.7 – hopefully, we’ll overcome the relevant hurdles in the next release or two.

The command-line client is still the reference implementation, and the GUI doesn’t expose every feature, but it may be more convenient for the cases it does support – do have a play. You’ll find it in the “bigv-gui” package for the Linux distributions we support, and it’s installed alongside the command-line client if you use the Windows installer. Speaking of packages, our Linux ones should now work properly on Debian Wheezy (Ruby 1.8 or 1.9) and Fedora Core 19 (Ruby 2.0).

That’s not all we’ve been working on, of course – bigv vm shutdown and bigv vm stop now do different things (the former is graceful, like pressing the power button; the latter is like pulling the power out) and you can send a hard reset signal too (bigv vm signal –vm-signal=reset). You can also now rename VMs (bigv vm rename), delete accounts (bigv account delete), and run the client behind a HTTP proxy server (the –http-proxy option to any command). When creating or reimaging a VM, Debian Wheezy, Ubuntu Precise, and Windows Server 2012 have been added as options (we don’t have an Ubuntu Raring image yet!)

Finally, a bushel of bugs have been squashed, although I’m sure many remain. If you happen to spot one in the wild, please do let us know – especially if it’s to do with the graphical client – and we’ll do our level best to fix it. Otherwise, enjoy!

Comment...

BigV is now open for use by all Bytemark Hosting customers.

BigV is Bytemark’s “cloud” hosting service, and allows for flexible, self-service deployment of virtual machines with permanent discs. It has been in development since August 2011 and runs thousands of live domains even before its launch. It costs £10 per month for a 1GiB virtual machine with 25GiB disc space, and you can scale each VM up to 180GiB of RAM. It’s completely driven from a command-line, and developed in the UK by Bytemark’s excellent in-house development team.

So if you’ve got a single sign-on account (i.e. any Bytemark customer), you can now use our registration instructions to set up the client and start spinning up BigV virtual machines from the command line. Please let us know what you think of it!

There are a few features we’re still working on before we call it “launched”:

• disc I/O rates will be improved;
• billing and signup will be better-integrated into the command line client;
• we still need to specify a default Service Level Agreement (though we’ve agreed that with some customers already).

Once these are fixed, by April I hope, we will have big hoopla.

I’ve also opened our beta form to new registrations on a temporary basis, so if you’re not a Bytemark customer feel free to sign up. If you are a Bytemark customer but have forgotten your sign-on password (before today you’d only have used it for domains and credit card updates) you can request a reset from our panel front page.

Comment...

I recently rambled too quickly through my BigV design paper at the excellent London Perl Workshop 2012 the week before last. If you wanted to fill in some of the handwaving, or just see what changes we’ve made to BigV’s architecture since March, I’ve uploaded the new paper here:

Design and Implementation of BigV (November 2012)

Comment...

Just over two years ago I put pen to paper on BigV‘s specification. It now hosts hundreds of live sites: citrulu.com for web site monitoring, luzme.com for up-to-the-minute eBook price comparisons, folktunefinder.com helps you “name that tune”, itsdanbull.com has rap and activism, quizible.com for quiz obsessives, bullfarmoast.co.uk is a lovely B&B, Tilley and Grace sell jewellery, and has weathered links from McFly… it’s a very long list. It gives me a lot of pleasure to flick through the hostnames and see what sites our testers have entrusted to our new platform. There are machines running Windows, machines running OpenBSD and of course, lots of Linux too. Even though we’ve had our odd hour or two of down time, BigV seems more reliable and more transparent than some “finished” VPS and cloud products out there, judging from the last 24 months at least.

The official launch won’t be for another couple of months. But we’re about to pass a big milestone: turning on our billing system, and actually collecting some money. By taking money for BigV while it’s a stable hosting platform, we can complete empty our beta test list and rapidly ramp up production of new features: disc snapshots, archiving, popular API support and everything else that’s queued up for “after the launch”.

To that end, we’ll create accounts on our billing system for all of our testers. If that’s you, you will get some instructions in the next few days on how to pay for your BigV usage. In exchange, we’ll publish (and stick to) our Service Level Agreement, and you can finally point your customers at a serious, paid-for service.

We will start emailing bills at a 100% discount during September, and start charging our full published rates from October. Testers who signed up with us before August 2011 will receive a £60 credit; that’s our thank-you for being interested before we’d announced very much. I hope that everyone else will feel that they have had a lot of free, reliable hosting already.

So you should now delete any VMs you don’t want to pay for. Even if you don’t want to use BigV for now, you can keep your account reserved without charge, assuming there are no active VMs on it, and come back at any time

Our billing system isn’t yet complete, and will charge on a daily resolution, so a minimum charge of 33p for creating a virtual machine. That’s why it’s still a beta test, although one that’s being wound down very slowly.

This puts our full launch into late October/November; in the mean time, please report anything you think is unusual, surprising or incorrect about the bills you see. We’ll be watching them too! And please also let us know if you have any particularly exciting sites or demonstrations that are using BigV in unexpected ways, because we’d love to feature your sites on the revamped bigv.io.

Happy B-day! (billing that is)

Comment...

There’s a new release of BigV’s client software – hurrah! So you are now able to “log in” or regenerate your profile on more than one machine. You can change your password rather than have to copy your .bigv folder around when (re)installing. The SSH key management is back, so you shouldn’t keep getting those prompts about host keys. You can create a VM with a CD-ROM ready inserted to speed installations.

More significantly you can also now create and delete usernames for other people – this means:

• you can use those handy privilege delegation commands to share a BigV account  with other people;
• any users you create can log in and create their own BigV accounts that are nothing to do with you.

So if you have friends who would like BigV accounts, you now have the power to provide them with one.

There were also a two things I’d forgotten to document: did you know you can incorporate bigv invocation into other programs through YAML and batch modes? Or that there is an interactive mode which saves you entering your password every time you run bigv? They were always sat there in the interactive help, but I’d not written them up until now.

Update instructions are as usual: if you’re using Debian or Ubuntu you can “apt-get update; apt-get install bigv”. Everyone else, go to http://bigv.io/download to grab the latest tarball or installer. There are some Redhat / Fedora / CentOS RPMs that I’ve not fully tested but I hope to post in the next fortnight. Thanks to Jamie Nguyen for doing this work for us.

Finally, Bytemark is turning 10 years old this year; I should soon have good news about the end of BigV’s beta period, and a launch this summer.

Comment...

Here’s the paper I wrote for last week’s FLOSS UK Spring 2012 conference. I don’t really do slides, just scrolling through text and pointing at diagrams. But hopefully this is more useful afterwards – the paper spills all the details on how we’ve been implementing BigV, our excellent new hosting system. Have a read, and I’d be happy to answer any questions in the comments.

I’ve also dug out the paper I did for the UKUUG 2004 Bournemouth conference on what we were doing with User-Mode Linux. There’s actually plenty there that’s still relevant to planning a virtualisation system – everything from  TAP interfaces to economies of scale. Both papers are a combination of half-imagined technology & business, but we did OK the first time round, so I’m thinking we have a shot with BigV!

Also at the UKUUG conference, I was fascinated to learn of Iustin Pop’s work on Ganeti, a VM supervision system developed at Google. If you’re looking to deploy a cluster of VMs on your own hardware without the bureaucracy of Openstack or Eucalyptus, I’d suggest you look at that; I certainly will be. If you do just want hugely flexible VMs with up to 8 discs and  120GiB RAM on our well-managed infrastructure, BigV is clearly where it’s at

Here are the two papers:

• Design and Implementation of BigV
• Getting the Best From Your Server with User-Mode Linux

Comment...