Using HDD with sizes > 2 TB (Internal or USB ) and SDHC/SDXC cards

Discussion in 'Functions Operation Guidance' started by Nice Monkey, May 24, 2017.

  1. Nice Monkey

    Nice Monkey Active Member


    OK finally figured out how that limit of > 2TB HDD support actually works and what can be done and what can not be done to overcome limitations.
    Everything mentioned here was implemented by myself in practice. Practical experience includes using 2, 3, 4, 5, 6 and 8 TB drives.

    After reading this tutorial you will see that HDD > 2TB is supported both as Internal HDD and as USB attached HDD. There are however conditions to this and formats under both conditions in fact are (mostly) conflictive with each other.

    I am afraid this is a bit technical, but as many of us including support staff was often confused about things, I will try to make it clear(er). If you are only interested in the do's and don'ts to make it work you may jump straight to Part-2.

    As a start one needs to understand the difference between "HDD Initiation" and "HDD Formatting".
    Often this is done in one step by a program, but the technical functions are different.
    - The "HDD Initiation" tells the interface driver where and how to Boot, how the HDD can physically be addressed and where the physical partitions can be found on the disc. Here Microsoft Windows MBR or GPT initiations will be used to be compatibel with various Media Players
    - "HDD Formatting" puts on the filesystem. This defines how and where files are put on the disk using as specific filesystem. Here the filesystem is assumed always to be Microsoft Windows NTFS to remain compatible with most Media Players, FAT32 works functionally but has a 4 GByte individual file size upper limit. That won't do for bigger MKV files and above all ISO images of DVD and BD. BD in fact will hardly ever work.

    Some technical background for the interested
    - First there are the Hard Drive manufacturers which make HDD drives traditonally read by sectors of 512 bytes each. With modern discs in Terabytes that is a lot of sectors which is not very efficient any more. Filesystems like FAT32 and NTFS therefore always read/write bigger blocks of data called clusters. A cluster is by default 4 KBytes (=8 sectors) and is configurable.

    - Sectors are referenced to using 32-bit registers. If one calculates right one learns that a maximum 2.2 TByte can be addressed with a 32- bit counter adressing 512 byte sectors. Changing that is far from simple as this resides deep in the OS and Drivers. Partioning does not help here as still the whole disc space needs to be accessed at the physical level. This was up till recent done via the Master Boot Record (MBR) and Logical Block Addressing (LBA) using 32-bit. Windows changed to GPT (GUID Partion Table) with Windows-7 using 64-bit addressing solving this limitation structurally.

    - But even with GPT the read/write overhead remains the same. So Hardware manufactures did a trick and intruduced Physical and Logical sectors. The newer 2 Tera Byte and bigger discs all work with Physical Sectors of 4K Bytes presented to the software as 8 Logical sectors of 512 bytes. This technology is called "Advanced Format 4K" which reduces the disc internal overhead but not the total CPU processing chain for I/O. The drives should be labeled 512e which stands for 512 byte logical sector emulation. To make a 512e HDD efficient sectors must be aligned at 4K boundaries. See the matching AF logo for a 512e HDD.

    - The next step is Native 4K support. See attached logo for these 4Kn HDD. With 4Kn drives Logical Sectors, Physical Sectors and Clusters are all identical 4K Bytes streamlining throughput that way. This requires drivers dealing with 4K Logical sectors and obviously bolts on support of GPT for Initiation. Windows supports Native 4K HDD starting with Windows 8.1 OS. This paves the road for hazzlefree support of > 2 TB and even > 16 TB HDD combined with improved performance. Starting begin 2016 the first Native 4K HDD's made by WD appeared on the market. These are sold as HDD or build-in into their USB3 Enclosures. Drives can easily be identified as both Logical Sectors and Physical Sectors are always 4K.

    You can display all mentioned HDD format details using fsutil fsinfo ntfsinfo x: under Windows. This is a CMD to be executed with Admin rights! (Note x: replaces the actual drive being used)
    You may also use "MiniTool Partition Wizard Free" (see bottom for more details).

    Back to most media players around
    Access can run via Internal HDD, USB HDD Docking Bay, USB HDD enclosure or a NAS.
    - A NAS setup hides the total physical drive access plus format (RAID) used and as such supports > 2TB drives without the Media Player even being aware of it.

    - Internal discs are SATA attached (like internal drives in your PC). Drivers for SATA are easier to adapt and most media players upgraded to GPT support. They can address > 2TB discs using Advanced Format 4K support. All external drives for mounting into a PC purchased in shops are formatted that way. There is always a buffering/memory limitation somewhere but there is not a hard limitation for the maximum disc size. We know that 6 TB and 8 TB drives do work, but I do expect drives far above 16 TB to work too.

    - USB attached drives is a complete different ballgame. Drivers are in all places and often very old. Also 64-bit addressing support may be jeopardized by a structural HW limitation. So GPT support is often not yet available and worse not being planned for many products. Under the condition that GPT is handled by products on both ends then USB standards do not impose limitations. In practice only some recent USB3 Docking Bay implementations do include such support.
    As a temporary fix hardware manufactures came up with a trick. They started to ship USB-enclosures (= Casing, USB/ATA bridge and a HDD as one unit) with native 4K Logical sector enabled. The USB/ATA bridge firmware presents these to the USB-side. Result MBR can now work with 32-bit adressing of 4K Logical sectors. The physical disk access is not a problem as this is done by the USB/ATA bridge here. This trick will obviously only work up to a limit of just above 16 TB capacity.

    Attached Files:

    • AF.gif
      File size:
      7.1 KB
    • 4Kn.gif
      File size:
      7.1 KB
    Rita likes this.
  2. Nice Monkey

    Nice Monkey Active Member


    Moving a HDD between Internal <=> External <=> NAS <=> Windows PC

    Please use Microsoft Windows 7/8/10 only for re-formatting with GPT+NTFS and have it (E)SATA attached for Internal HDD's. I am sure that will work. MAC and Linux equivalents may do the trick, but never tried those myself. Nobody ever reported (good) results for media players
    Internal HDD and USB HDD > 2TB are not inter-changeable and need to be re-initiated and re-formatted for each deployment. As most people don't open USB Drives/Housings/Enclosures they will never be aware of the difference. Personally I am now able to move any HDD in either direction without problems. This with the exception of (still exotic) 4Kn type HDD's having 4K Physical sectors not handled by Android players.

    - A HDD pulled from a USB Drive/Housing/Enclosure will work only as an internal SATA attached drive again after standard re-initialisation + re-formatting. Obviously this HDD must be (E)SATA attached during mandatory re-initialisation + re-formatting using Windows-7 or equivalent standard tools. A common reason for doing so is that a complete USB-Drive may factually be cheaper than buying the same HDD as a separate unit. You still may be able to re-use the then empty USB-External enclosure later.

    - A HDD pulled from a Plug & Play USB3 Docking Bay with GPT support (like Fantec's, Icy Box and probably Lindy's) will work straight as an Internal HDD again. In fact you may move individual units freely between such a Docking Bay and Internal bays of multiple brands. Tried this with sizes up to 8 TB. This won't work with RAID supporting variants obviously as these will need a re-build of many hours when a HDD is changed (even in JBOD mode).

    - Formatting with 4K Logical sectors for USB External drives requires special tooling, it also helps if you exactly know what you are doing using such tools. I am using tooling from Acronis for this just as most USB HDD manufacturers do. You will run into this whenever you want to insert a new/re-used standard SATA internal HDD into an existing External USB Housing e.g. to increase its capacity.

    - If you pull a HDD from a NAS then it won't work for either deployment as it has been re-formatted the wrong way. It may be using EXT3/EXT4 but in most cases will be using a proprietary format. Re-format it first for the purpose you have in mind being Internal or External USB enclosure as explained.

    - A HDD pulled from a working Windows PC might still not work despite being formatted correctly with GPT + NTFS. The reason is Windows and/or the manufacturer has put an additional (hidden) partition on it for OS recovery purposes. Media players like to see a single Partition starting at the beginning of the physical disk. You need to remove all existing partitions first and next format again with GPT + NTFS when (E)SATA attached. Doing so via USB won't work in most cases neither as resulting HDD parameters are slightly different also not interpreted by Android.

    Using (EXT2/EXT3) EXT4 formatted HDD in combination with MAC or Unix/Linux/Android based platforms.
    First of all this is not officially supported by some media players but it is validated to be working just fine. This does not come as a real surprise as Android is a Linux derivation and Android kernels include support for EXT2/3/4.
    EXT2 is often used on USB-sticks and SD-cards, but is less suitable (=less robust) for deployment on a HDD use EXT4 by preference.

    Obviously EXT formatting has to deal with the same HDD technical developments as NTFS:
    - EXT3 will support a HDD up to 16 TB using a Logical 4K block size The choice of the block size to be used may be automatic or a variable to be set manually; this depending on the OS, OS-version and the factual tool being used.
    - EXT4 has no implicit HDD upper limit regarding total volume. Per OS and OS-version a support limit may be applicable and specified by the manufacturer. In practice EXT4 will work with any size HDD available within the foreseeable future.

    SDHC/SDXC Cards and using exFAT for HDD?
    exFAT is the new defacto File System standard for SDXC cards. In fact exFAT was designed for memory cards and sticks and not for HDD. It is an excellent replacement for FAT32 removing all its practical limitations but not an advisable (=less robust) replacement for NTFS.
    As SD-Cards are immensely popular this will move exFAT to almost any platform. You see already a lot of devices (above all phones) with SDXC/exFAT support included. Media Players are very slow to adopt native SDXC card support. Using exFAT on SDHC cards should work in normal cases as it has become a standard Android Lollipop feature by now.

    Occasionally I read that only exFAT support makes the difference between SDHC and SDXC, this is a mistake as support for the new UHS-I bus is the real technical innovation coming with SDXC. This new SDXC bus has a far higher speed . The UHS-I backward compatible UHS-II bus will again double throughput and is almost as fast as SATA600 and has 2 rows of pins making identification easy.
    Some brands call everything up to 32 GByte SDHC (with FAT32) and above 32 GByte SDXC (with exFAT) which is misleading. I have seen 128 GByte SDHC cards not supporting the UHS-I bus. Look at the corresponding UHS-I/UHS-II speed labels which must be present for SDXC compatibility and performance indication. Media players will not work with native SDXC cards! Never seen any UHS-I bus enabled media player yet which is a pity.

    SDHC/SDXC Hybrid Cards which also exist should always carry both a SDHC (e.g. @10) and SDXC (e.g. U1) speed indicator as they work when inserted in old SDHC slots and new SDXC slots. In reality they may not always carry both labels, specifically SANdisk is known to make a real mess with labeling. See the attachment for a correctly labeled Kingston card. The term hybrid card which I use here is never used for any SD-card advertising which adds to the confusion.
    When you e.g. put a 32 GByte Hybrid card (mostly labeled as SDHC) into a card-reader for SDXC with UHS support, then it will be blazing fast. The reverse is also true, putting a Hybrid card of 64+ GByte into an SDHC card-reader slot will slow it down to @10 speeds maximum even being an ultra fast UHS-II card. This slow down will always happen when using such a hybrid card with media players as it is run in compatibility mode then.
    When using a SDXC capable external USB Card Reader then it must be USB3 connected as transfer speeds are way above USB2 limits. Measured 85 MB/sec Read and 30 MB/sec Write on a 12 Euro 32 GB Hybrid card. This card works also with most of my media players. One player (Xtreamer eXpress) refuses to read hybrid cards for an unknown reason.

    If you want to validate/view very detailed HDD settings yourself and/or to try various formats suitable for Media Players then you can use the excellent Windows based "MiniTool Partition Wizard Free 10.2".
    It allows HDD formatting with EXT2/EXT3/EXT4 using Windows too

    Attached Files:

    Last edited: May 24, 2017
    Rita likes this.
  3. Jean-Michel

    Jean-Michel New Member

    Je viens de recevoir un A10 avec un disque dur pour 4D WD NAS RED et j'ai confirmé qu'il n'est pas reconnu par l'A10, je l'ai renvoyé à AMAZON pour le remplacer par un WD 4To vert; comme indiqué plus haut j'aurais pu le reformater, mais je viens du renvoyer.
    Merci votre site est bien fait, avez-vous un modérateur, car je viens de lire un message totalement déplacé sur un autre chapitre il y a 10 minutes.
  4. Rita

    Rita Administrator Staff Member

    Hi @Jean-Michel , please tell us what's your problem in English? Thanks.
  5. Nice Monkey

    Nice Monkey Active Member

    Also a WD Red should be recognized by Egreat players. No need to replace it with a Green edition.
    If is not then it needs to be re-formatted. First remove all partitions (including any hidden) and next re-format it as documented.
  6. Gaara80

    Gaara80 Member

    Hi, I have connected the a11 to the network (i have a wd red 6tb into it) with a router that have gigabit port. Form the iMac i see the egreat in network but if i trasfer a 38gb file from mac (that have gigabit port of course) to egreat it is very slow like 50minutes and this is impossible because i have a gigabit router and mac. Does the egreat have a bug in the ethernet???
  7. Nice Monkey

    Nice Monkey Active Member

    Don't have a NAS myself anymore so I can't test it for you. What are you using SMB, FTP or NFS?

    It has sub-optimal parameters for the Samba (=SMB) server. These can be updated using a PC connection though. Just look for it. This is when using the Egreat as a NAS which I do use occasionally (there is another topic about that).
    Last edited: Jan 24, 2018
    Gaara80 likes this.
  8. Gaara80

    Gaara80 Member

    Well from the mac in the fidnder menu after the command "cmd+k" i wrote smb://egreat ip address an then i saw the share folder with two other folders sda 1 and sda2 but i think it is only 100mb/s. I have also a pc what can i do???
  9. Jean-Michel

    Jean-Michel New Member

    Sur mon A10, j'ai remplacé le DD 4To red, par un WD 4To Blue, mais auparavant il a fallu le formater en NFTS pour qu'il soit reconnu, j'ai pu charger mes premiers films de mon camescope 4K Sony FDR AX100E et le résultat est parfait sur son téléviseur 4K Sony.
    Jean Michel

Share This Page