[NTLUG:Discuss] SCSI performance question

[Robert Pearson]

On Sat, Oct 25, 2008 at 4:20 PM, Robert Parkhurst
<robert.parkhurst at gmail.com> wrote:
> Ah!  Sorry for not including that information earlier!  I'll try to answer
> as best I can..
>
> The Specs / RAID layout / Drives (etc.) are as follows:
>
> If I were to go with JetStor, it's got a PCI-X (133MHz) LSI RAID controller
> with 256MB cache on it.  The drives are SATA-II (with U320 interface on the
> backplane that would be used to connect back to the Linux "head node").  The
> enclosure has 16 drives and would be configured for RAID-5 or RAID-6 for
> protection...  (Although you could also configure it for other RAID's like
> RAID-0).  The unit comes with two U320 LVD connections on the back side.
>
> The URL is:  http://acnc.com/02_01_jetstor_sata_416s.html
>
>
> The Application / Usage:
>
> This storage system would be used in a litigation support company.  They use
> (windows) software that "cracks" files--the software opens up the files,
> extracts the data and meta-data from them, and creates files from that.
> Each file gets a tiff image made along with a meta-data file..  So for
> example a word document becomes three documents:  the origional word
> document, the tiff image and the metadata file for the tiff image.
>
> There's LOTS of I/O on the system, both reads and writes..  The process is
> this:  You get the client data that needs to be cracked for use in the
> case..  You save that data to the server, then run the "cracking" software
> against it.  The cracking software reads that origional source data, cracks
> it, then saves the new files out to a destination (also on the server).
>
>
> Right now they've got a single Linux box with an internal RAID array using
> an Areca RAID card to pass through all the drives, then Linux does software
> RAID-5 (I asked if they wanted me to make a hardware RAID out of it, but my
> boss at the time didn't want to have to rebuild the raid....).  Anyway,
> that's shared out to Windows via SMB and the server performs SO well that
> ALL work is now done on that server alone vs. before when they had to
> distribute the load between multiple windows servers becasue no single
> windows server could handle all the reads and writes.   So it got me
> thinking that if we went to direct SCSI attached storage, using LVM + XFS
> and such we could expand the RAID and still keep the performance that the
> Linux box gave without having to throw $$$ at some place like Dell or
> NetApp.
>
> My concern though for doing this would be what are the limits of SCSI in a
> configuration like this -- putting 31 x JetStor's (RAID-6) on a single PCI-e
> (x8) LSI SCSI Card.  Would it be possible to still get the high performance
> or would it be best to limit it to something like 8 x JetStor enclosures or
> something?
>
> Hope this helps and again, thanks for any input!
>
> Robert
>

> On Sun, Jul 5, 2009 at 9:10 PM, Leroy Tennison
> <leroy_tennison at prodigy.net>wrote:
>
>> Robert Parkhurst wrote:
>> > I've used SCSI a lot, but mostly in the "lower-end" area (external CD/DVD
>> > SCSI drives, external SCSI hard drive or even a small SCSI external disk
>> > pack).  I'm curious though how SCSI would perform if you had a lot of
>> large
>> > disk arrays attached off a single SCSI bus/adapter?
>> >
>> > Specifically, say I had an LSI SCSI adapter (Ultra-320, PCI-e (x8)) that
>> > could have 32 SCSI devices attached to it.  And I attached 31x16TB
>> external
>> > SCSI enclosures (like the JetStor SCSI unit or something) off that one
>> > adapter and then used Linux to make an LVM striped volume over all of
>> them
>> > and formatted it with something like ReiserFS or XFS.  Would I see a
>> > (noticeable) performance hit on it?
>> >
>> > And if so, what's a good "recommended" max for attaching storage like
>> that
>> > to a single SCSI controller?
>> >
>> > Thanks in advance!
>> >
>> > Robert
>> > _______________________________________________


>> This is almost impossible to answer because performance is affected by
>> too many variables: what are the capabilities of the SCSI controller
>> (caching and how much? is the firmware efficient? how many channels?
>> other performance characteristics), the SCSI drives (all the common hard
>> drive performance characteristics), how is it configured (RAID?  If so,
>> which level?) and what your application is (mainly read-only, lots of
>> writing, lots of deletions after writing, a database).  Further
>> complicating matters is the fact that it's not just individual issues
>> but combinations of them (matching the configuration and the hardware to
>> the application).
>>


"the (Linux) server performs SO well that ALL work is now done on that
server alone"
Why is faster Storage needed?
"single Linux box with an internal RAID array using an Areca RAID card
to pass through all the drives, then Linux does software RAID-5"
This configuration might reconfigured for better performance and
Information Integrity. A lot of work for little gain...
"shared out to Windows via SMB"
SMB from Direct Connect Storage on a server doing file cracking could
be a major bottle-neck.
Perhaps the SMB Storage could be on the network? NAS or SAN? SATA
drives would be the choice here.
Crack on the existing server and Storage and de-duplicate the
"cracked" files to the network Storage.
The goal is to keep the "cracked" files stored on the "cracking"
server to a minimum.
Typically "cracking servers cannot have too much memory. Faster
processors make a huge difference.