How to Break the 2TB (2 TeraByte) File System Limit
Posted by Carlton Bale on 28th May 2007
In the 1990's, breaking the 640 KB memory limit of DOS was a big deal. Now, with single hard drives reaching capacities of 1 TB (1,000 GB), it is very easy to create a RAID array larger than 2 TB, but your operating system may not be able to handle it. But if you don't plan ahead, your operating system will only be able to address the first 2 TB and all that extra storage beyond 2 TB will be unusable. Here is an overview of some of the methods you can use to get around the 2 TB limit.
Equipment Requirements:
- Hardware RAID controller capable of 64-bit LBA addressing (for volume sizes greater than 2 TB). For this example, I'll use an Areca ARC-1230 RAID card.
- Several hard drives to connect to the RAID controller to create a RAID array. For this example, I'll assume (five) 1000GB SATA drives.
- Drives must be configured in a RAID level 5 Volume Set. For the first two examples, I'll assume all five drives are members the same RAID level 5 volume set. RAID level 5 requires the space of 1 drive to be allocated for parity data, so total available storage space for 5 drives will be 4 drives x 1000GB = 4TB.
Required Background Information: I'm assuming you already have an understanding of RAID 5, its benefits, and requirements. If not, read this Wikipedia article. Now, let's discuss the difference between RAID sets, Volume Sets, and Operating System Partitions.
- RAID Sets are groups of drives that a RAID controller groups together to act as one single array. The individual disks are not visible to the operating system but rather are controlled by a hardware RAID controller.
- Volume Sets are create by the RAID controller and reside on top of RAID Sets. A Volume Set set is presented to the operating system as a single, virtual disk drive. This is a little confusing, but the RAID level (RAID level 5 in this example) is determined when the Volume Set is created (not when the RAID Set is created.) It is possible to have multiple Volume Sets residing on the same RAID set, and the Volume Sets may even use different RAID levels.
- Partitions are created by the Operating System and reside on top of Volume Sets. (Volume Sets appear as virtual disk drives to the operating system.) You can use have the Operating System create one or more formatted partitions on top of a volume set.
Breaking 2TB Option 1 - Use Windows with NTFS and GUID Partition Tables (GPT) partitions. It is possible for Windows to use NTFS partitions larger than 2TB as long as they are configured properly. Windows requires that the GUID Partition Tables be used in place of the standard Master Boot Record (MBR) partition tables. You will need Windows XP x64 Edition or Windows Server 2003 Service Pack 1, Windows Vista, or later for GPT support. (It is possible to mount and read existing GPT partitions under Windows XP and 2000 using GPT Mounter from Mediafour.) There are a couple of stipulations for GPT disks. First, the system drive on which Windows is installed can't be a GPT disk because it is not possible to boot to a GPT partition. Secondly, an existing MBR partition can't be converted to GPT unless it is completely empty; you must either delete everything and convert or create the partition as GPT. Read this Microsoft TechNet article for more details on GPT.
- To summarize: 1 RAID array of five 1TB Drives -> 1 RAID level 5 Volume Set that is 4TB -> 1 NTFS GUID Partition Table Windows partition that is 4TB.
Breaking 2TB Option 2 - Use Linux with CONFIG_LBD enabled. Most Linux file systems are capable of partitions larger than 2 TB, as long as the Linux kernel itself is. (See this comparison of Linux file systems.) Most Linux distributions now have kernels compiled with CONFIG_LBD enabled (Ubuntu 6.10 does, for example.) As long as the kernel is configured/compiled properly, it is straight-forward to create a single 4TB EXT3 (or similar) partition.
- To summarize: 1 RAID array of five 1TB Drives -> 1 RAID level 5 Volume Set that is 4TB -> 1 EXT3 (or similar) Linux partition that is 4TB.
Breaking 2TB Option 3 - Use Standard Partitions and Create Multiple Volume Sets within a RAID array. A RAID array itself can be larger than 2 TB without presenting a volume set larger than 2 TB to the operating system. This way, you can use older file systems (that support only 2TB) and still have RAID 5 protection and more than 2 TB of total storage. To do this, put all 5 drives into a RAID set and create a 2 TB RAID Level 5 volume set — this will leave 2TB of the RAID set unused. Then create a second 2 TB RAID level 5 volume set. Boot into your operating system, create a partition on each of the 2TB virtual drives, and format each of the two 2TB virtual drives. The disadvantage is that there is not one single, large 4TB partition. The advantage is that 1) backwards compatibility for the file system and partitions and 2) they are both part of a RAID 5 array and are protected from single drive failures and only 1 drives worth of storage is sacrificed for RAID parity data.
- To summarize: 1 RAID array of five 1TB Drives -> 2 RAID level 5 Volume Sets that are 2TB each -> 2 standard NTFS (or any other) partitions that are 2TB.
Note 1: RAID Capacity Expansion. If your RAID card supports online capacity expansion, it is possible to expand any of the configurations above. For options 1 and 2, expand the RAID Set, then Expand the Volume Set, then Expand the Operating System partition. For option 3, expand the Raid Set, Create a 3rd Raid level 5 Volume set, and then create a third operating system partition. To learn more about expanding a RAID array on an Areca controller running Windows, read this article.
Note 2: Software RAID. Software RAID adds an additional level of complexity to RAID. For that reason, I recommend using a Hardware RAID controller. Having said that, I think everything mentioned above is technically possible if you are using software RAID, but I've never messed with it some I'm not positive.
Posted in Computer Hardware and Software, Gadgets | 18 Comments »