fileserver

In computing, a file server is a computer attached to a network that has the primary purpose of providing a location for shared disk access, i.e. [read more at http://en.wikipedia.org/wiki/File_server]

  • NAS are very popular these days; for people who can not afford a professional NAS or a sub 1000€ system, the idea of recycling an old pc come quickly to mind. What is a little bit more difficult is to find the right hardware and software combination.
    I higly recommend Promise SATA raid controller for their Linux native support, while for software I came across OpenFiler; an open source project which has professional functionnalities...

    Openfiler is a powerful, intuitive browser-based network storage software distribution. Openfiler delivers file-based Network Attached Storage and block-based Storage Area Networking in a single framework. Openfiler sits atop of CentOS Linux (which is derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor). It is distributed as a stand-alone Linux distribution.

    The entire software stack interfaces with third-party software that is all open source. File-based networking protocols supported by Openfiler include: NFS, SMB/CIFS, HTTP/WebDAV and FTP. Network directories supported by Openfiler include NIS, LDAP (with support for SMB/CIFS encrypted passwords), Active Directory (in native and mixed modes) and Hesiod. Authentication protocols include Kerberos 5. Openfiler includes support for volume-based partitioning, iSCSI (initiator), scheduled snapshots, resource quota, and a single unified interface for share management which makes allocating shares for various network file-system protocols a breeze.

    Note that I am currently thinking on buying a ReadyNAS NVbecuse of its size
  •  I will be building in the next few days my own Network Array Storage (NAS) monster:

    • RAID 6 , instead of RAID5, so 2 disks can fail in the array. A 7 Hot spare is planned
    • Hardware RAID because the cheap motherboard (NVIDIA) only support in best case software RAID5
    • GIGABIT network...
    • Linux powered of course: FreeBSD for security or OpenFiler (RedHat) for it's ease of use...
    • Crypto loop (Linux kernel 2.6 dmcrypt), private keys on USB stick, only 20GB will NOT be encrypted
    • FileSystem: XFS or ReiserFS because I will be storing big files

    The hardware will look as follow:

    • 6 Maxtor 7L300R0 MaXLine III, 7200rpm, 16MB, 300GB, IDE, 24/7 server : 60 Months garanty! 104€/each = 624€
    • AMD Athlon 64 3000+  BOX, Socket 939, Venice, the least expensive Athlon 64     99€
    • Asus A8N-VM CSM, mATX, Nvidia 6150/430 Video, socket 939, SATA RAID, because I need PCI-E for the hardware RAID  70€
    • Promise SuperTrak EX8350, SATA2, 8 SATA port, Raid6 Controller, because Promise support Linux   297€
    • Thermaltake Armor (bought previously)
    • Zalman CNPS9500 LED, Socket 754/939/940/478/LGA775  53€

    I will explain later why I did not bought a SOHO NAS, but briefly because:

    • For the price of my complete system, I have an empty SOHO NAS box or with 250GB (Raid0)
    • I have 8 + 6 = 14 SATA ports!
    • SOHO NAS are only RAID5 in best case
    • Bad performances of SOHO NAS
    More pictures, howto, and benchmarks soon...
  • origin: WikiPedia

    SAMBA
    An open source implementation of the SMB file sharing protocol that provides file and print services to SMB/CIFS clients. Samba allows a non-Windows server to communicate with the same networking protocol as the Windows products. Samba was originally developed for Unix but can now run on Linux, FreeBSD and other Unix variants. It is freely available under the GNU General Public License. The name Samba is a variant of SMB, the protocol from which it stems. As of version 3, samba not only provides file and print services for various Microsoft Windows clients but can also integrate with a Windows Server domain, either as a Primary Domain Controller (PDC) or as a Domain Member. It can also be part of an Active Directory domain.

    CIFS
    Server message block (SMB) is a network protocol mainly applied to share files, printers, serial ports, and miscellaneous communications between nodes on a network. It is mainly used by Microsoft Windows equipped computers.

    FTP
    The File Transfer Protocol (FTP) is a software standard for transferring computer files between machines with widely different operating systems. It belongs to the application layer of the Internet protocol suite.
    NFS
    Network File System (NFS) is a protocol originally developed by Sun Microsystems in 1984 and defined in RFCs 1094, 1813, (3010) and 3530, as a file system which allows a computer to access files over a network as easily as if they were on its local disks.

    RSYNC
    rsync is a computer program which synchronises files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.


  • Penguin computing power!

    Here we goes, I've receive yesterday all missing hardware to finish the building of my own NAS.

    A NAS (or Network Attached Storage) is a hard disk storage device that is set up with its own network address rather than being attached directly to the computer that is serving applications or files to a network's users. By using a NAS, both applications and files can be served faster because they are not competing for the same processor resources. The NAS is attached to a local area network (typically, an Ethernet network) and assigned an IP address....

    Here is some pictures...Nothing really special, if You already know how to build a computer by Your own...


    The 4 Western Digital Hard disks RAID optimized because they have been made to have a time-limited error recovery which improves compatibility with RAID adapters, and prevents drive fallout caused by the extended hard drive error-recovery processes common to desktop hard drives.

    "Desktop drives are designed to protect and recover data, at times pausing for as much as a few minutes to make sure that data is recovered. Inside a RAID system, where the RAID controller handles error recovery, the drive needn't pause for extended periods to recover data. In fact, heroic error recovery attempts can cause a RAID system to drop a drive out of the array. WD RE is engineered to prevent hard drive error recovery fallout by limiting the drive's error recovery time. With error recovery factory set to seven seconds, the drive has time to attempt a recovery, allow the RAID controller to log the error, and still stay online." from  Western Digital

    These drives along with the Maxtor Maxline III  have 60 months guaranty: highly recommended!

    Ask the shop to provide You disks not from the same batch, to reduce statically disk fallout.
    In order to replace a faulty RAID disk as fast as possible, it is not a bad idea to put a number on them. Normally hard disks in a NAS are into zero force and hot swap bays. But they cost at least 250$ for 4 drives...
    First batch of 3 disks (3 x 320GB Western Digital). Using a Thermaltake Armor tower help a lot in my setting.
    Same remark, it is obvious but cables can also fail, and it is not recommended to pull out the wrong cable if the array is online.
    Second batch of 3 disks, the number 6 will be mounted later.
    Thermaltake provide in the front bay a really good cooling fan with an integrated blue led.
    The Asus A8N-VM mainboard  mini ATX, Nvidia 6150/430 Video, socket 939, SATA RAID 0,1,5

    Ohhh no, cables are starting to pop out the case. Do not expect to see  a Mackintosh ordered internal case in the next few pictures...
    Routing cables, the case has a lot of possibilities to hide them.

    The AMD Athlon 64 3000+, Socket 939, Venice core, is cooled down by a Zallmann  CNPS9500
    Bringing power to all disks.
    Front of the case, the mainboard along with the additional hardware RAID card (Promise SuperTrak EX8350, SATA2) has more than 14 SATA ports...plenty of extensions possibilities with a case of...20 bays.
    The front cooler will be able to suck air freely.
    Power ON!

    The system is making a lot of noise (not only coming from FANs), I reduce the Zallmann noise (horrible at full speed) with the included speed controller.
    The 7 hard disks (one in hot swap) are making the case wobbling.
    Detail on the Zalmann CNPS9500 LED

  • Status: in development
    Developers: 5

    homepagewww.freenas.org 
     version0.66
    Based onFreeBSD 6
    SupportCIFS (samba), FTP, NFS, RSYNC
    Software Raid0,1,5 
    Hardware Raidyes if supported by FreeBSD 6
    InterfaceWeb interface, PHP scripts
    Size16MB
    Can be installedCompact Flash, hard drive or USB key
    FilesystemUFS, FAT32, EXT2/EXT3, NTFS (limited read-only)
    HardDriveATA/SATA, SCSI, USB and Firewire
    NetworkAll supported cards by FreeBSD 6 (including wireless card!)

    Added value
    Test it without breaking your NAS server with the VMWARE image:
    FreeNAS is installed on the first hard drive (2 partitions), with a RAID 5 volume for the 3 others hard drive. The IP address configured is 192.168.1.1, with default login/password.

    Why choosing itWhy avoiding it
    Small, do not need an additional disk for the OSFuture releases?
    16MB
    FreeBSD secure out the box: the least number of buffer vulnerabilities since years!
    Very nice GUI


    Performances Tests

    in progress

  • RAID @ home raid5Presentation

    Openfiler
    is a powerful, intuitive browser-based network storage software distribution. Openfiler delivers file-based Network Attached Storage and block-based Storage Area Networking in a single framework.

    Openfiler sits atop of CentOS Linux (which is derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor). It is distributed as a stand-alone Linux distribution. The entire software stack interfaces with third-party software that is all open source.

    Status: stable, in development
    # Developers: __


    homepage www.openfiler.com
     version 1.1.1 stable (2005)
    2.0beta (2006)
    Based on CentOS Linux
    Support
    • NFS,
    • SMB/CIFS,
    • HTTP/WebDAV
    • FTP
    • more
    Network directories support
    • NIS, LDAP (with support for SMB/CIFS encrypted passwords)
    • Active Directory
    • Hesiod
    • Kerberos 5.
    • more
    Software Raid 0,1,5,6
    Hardware Raid yes if supported by CentOS
    Interface Web interface, PHP scripts
    Size 4GB
    Can be installed On hard disk only because of its size
    File system UFS, FAT32, EXT2/EXT3, NTFS (limited read-only)
    HardDrive ATA/SATA, SCSI, USB and Firewire
    Network All supported by CentOS (including wireless card!)

    RAID @ home raid5  Installation

    Is straightforward, You only have to follow the flow on screen. But here is a small HowTo:

    RAID @ home raid5  HowTo: software RAID 5 install

    What I want: software RAID 5, 4 disk of 320GB (real 305GB), using NVIDIA SATA chipset (not a dedicated RAID5 board).

    Note: these pictures are not screenshots but picture taken with a sony camera...
    http://www.openfiler.com/download
    Download iso image from sourceforge and burn it to a CD Insert CD, and boot the PC
    The first step is to TEST the quality of the medium. Openfiler did not recognize the NVIDIA controller.
    choose "Add device"
    NVIDIA drivers (both Ethernet and drive controller) are at the end of the list.
    I've add both drivers manually. Then "Done" The welcome page. click "Next"
    Choose Keyboard language. I am not a novice, so let's look at the advanced configuration.
    The 4 disks are recognized. First I add some space for the Operating System. If You do not want to have a fifth disk just for the operating system, You'll have to reserve a small amount of the global space for the Openfiler system. Note: this space will be located on the first disk and wont be in the raid array...so no redundancy.
    Anyway it is uncommon to install the RAID engine on the RAID array itself.
    Lets have 2GB for System.
    And  1 GB for SWAP Then I click on th button RAID, since I have no RAID predefined only the first choice is available: "Create a RAID partition"
    I will have to create a RAID partition for each of the 4 drives, I reserved 300GB for disk SDB 300GB for disk SDC, for Disk SDA
    and for disk SDD...till all 4 disks contains a RAID partition. I click on the RAID button for the 5th times: and choose "create a RAID device"
    My disk array will be named /RAID (mount point), Raid level 5 Result, a /RAID (device /dev/md0) with an EXT3 file system.
    Nothing particular, default values are good Language support: English
    Choose TimeZone Enter a good Root password. Mine s too small but it is only a prototype for determining performances and reliability of the setup.
    Confirm all values entered by clicking Next wait till the raid array initialization

    CD get ejected, and reboot. Point your browser to https://box_ip:446/
    And administer remotely the box

    If all my explanations are not clear enough, or You want more details, visit the official installation page

    RAID @ home raid5  HowTo: hardware RAID 5 install

    in progress...

    RAID @ home raid5  Administration

    Check Openfiler Administration guide
    point your browser to https://box_ip:446/

    RAID @ home raid5  Problems encountered

    OpenFiler 1.1
    2.0beta1
    2.0beta2
    • The SATA controller was not recognized, this force me to use the 2.0Beta.
    • Unable to read or mount manually 2 different USB keys (FAT32), also unable to read CDROM (closed ISO and CDRW)
    • The network card (NFORCE 4 ) was not recognized by Openfiler 2.0Beta, I fail to copy the NVIDIA driver on the box because of point 2.
    • Is working perfectly, did not ask for any supplemental drivers

    RAID @ home raid5  Web Interface GUI

    Screenshots

    RAID @ home raid5  Performances Tests

    in progress


    RAID @ home raid5  Conclusions

    Why choosing it Why avoiding it
    Enterprise NAS features out of the box You do not need enterprise NAS features
    Very nice WEB GUI 4GB is too much and need an additional small disk only for starting the OS
    A lot of functionalities
    Limited choice of file system:
    • no Reiserfs, the swiss knife of all filesystem.
    • no JFS, XFS more adapted for big files
    A big communities of users and developers, good online documentation.
    No AMD64 version, but it's really not an issue.
    Very easy to have a software RAID5 arrays setup working.
    Stable, Linux 2.6.9 kernel base.
    GPL but an Enterprise version (with support) is also available.


     
  • in construction
  • in construction
  •  I've start looking at RAID 5 NAS array (a way to ensure redundancy of data and allow a set of files to be accessible across a network of machine) system 2 months ago, reading a lot of articles at the best hardware reviewers:

    A lot of new products have appeared in the last months, sign of a consumer demand. I have a lot of possibilities, each with their strengths and weakness:

    Infrant Ready NAS NV     Intel SS4000EThecus 4100

    1. Build my own small system, ($300 without disks), an ASUS Nforce mainboard (Gigabit, PCI-E, Video), an Athlon X64 3200MHz, but the CPU alone consume 90Watts (less in economic mode) and it is difficult to find a power supply under 200Watts. I already have a box (A mini ThermalTake tower for sure too big in the living room)
    2. Buy a Infrant  ReadyNAS NV ($900 without disks), because it has a great community (Forums), is small, look nice, consume only 50Watts. But I am concerned by performances problems (not consistent, good in read). Attention it is by far the faster SOHO NAS on the market as it outperform Buffalo Terrastation, Synologic base NAS by figures.  [AnandTech]
    3. Buy a Intel SS4000E ($850 without disks), mainly because its small, run dedicated XOR engine at 400MHz  vs only 200MHz for Infrant NV, but it also consume a lot more: 200Watts, and it hasn't been reviewed till now. Intel technical sheet also state that the CPU can reach 600MHz.
    4. Buy a dedicated RAID 5 hardware card, there is a lot available, but their prices are ridiculous for a personal use, more than 400 euro and for a little more it is today possible to build a top system based on a NFORCE4, Athlon XP64, Memory. Linux driver support is not bad (Promise, Escalade) but their drivers are not open source. This option fell down, as I do not have a PCI-Express port on my A7N8X NFORCE2, and may want to get rid of that big tower soon.
    I also want to have a LINUX powered NAS, because I feel more confident with Linux file system, where filename case is relevant, kernel can get stripped down to what it really need, and do not require a costly license (Windows XP or Embeded 2003 are out). I found a lot of  open source and free RAID operating system:  OpenFiler, FreeNAS, NasLite for naming a few.

    I came also across some very good resources, one for example listing the SATA chipsets which are recognized under Linux which is a must read before buying any mainboard or controller. And then get shocked by this performance RAID roundup: Hardware Vs Software RAID, where the Linux kernel is a clearly winner.

    Basically, the number of choice are now limited:
    • Wait for the Intel SS4000E review, or hope for a faster ReadyNas from Infrant.
    • Keep my biggest tower (huge Thermaltake Armor) and run on a new mainboard (Time to get rid of my 2001 mainboard NFORCE2? ) a software RAID array.
    I expect to build a Linux NAS Raid 5 array  made of  4  Maxtor 7L300S0 MaxLine III, 7200rpm, 16MB, 300GB, SATA, 24/7, 1M MTBF(5 years garanty) as I already have 2 of them and found them reliable, for a total of  3/4 * 1200GB = 900GB of raw data, and hook  to it 2 external USB disks (OneTouch 250GB and OneTouch2 250GB).

    Links and resources