Discussion:
[lopsa-tech] MS Windows as NFS Server?
(too old to reply)
Atom Powers
2014-05-19 21:06:27 UTC
Permalink
As an avenue of deprecating a NetApp device we are investigating a
block-only SAN device. That would mean using MS Windows 2012R2 to publish
shares to clients: MS Windows, Mac OSX, Linux. The MS Windows Admins manage
these file shares so Linux w/ Samba isn't an option.

Does anybody have any insight into using MS Windows as a file server for
NFS shares?
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
Will Dennis
2014-05-19 21:38:12 UTC
Permalink
Looks like “CIFS” (actually SMB v2/v3) would be your best share-publishing protocol here
 OS X and Linux have good support for this at this point, and why make Windows Server use a not-as-well supported protocol if you don’t have to?

If you must use NFS, looks like there is support in the product (don’t know about 2012R2, but assume the same as 2012
) - http://blogs.technet.com/b/filecab/archive/2012/09/14/server-for-nfs-in-windows-server-2012.aspx

W.

From: tech-***@lists.lopsa.org [mailto:tech-***@lists.lopsa.org] On Behalf Of Atom Powers
Sent: Monday, May 19, 2014 5:06 PM
To: LOPSA Tech
Subject: [lopsa-tech] MS Windows as NFS Server?

As an avenue of deprecating a NetApp device we are investigating a block-only SAN device. That would mean using MS Windows 2012R2 to publish shares to clients: MS Windows, Mac OSX, Linux. The MS Windows Admins manage these file shares so Linux w/ Samba isn't an option.

Does anybody have any insight into using MS Windows as a file server for NFS shares?
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
Robert Hajime Lanning
2014-05-19 21:44:19 UTC
Permalink
Looks like “CIFS” (actually SMB v2/v3) would be your best
share-publishing protocol here… OS X and Linux have good support for
this at this point, and why make Windows Server use a not-as-well
supported protocol if you don’t have to?
If all the clients are desktops with single users, sure.

I would hate to manage a multi-user client or a server that needs to
mount everyone's home directory. One mount/connection per user...
--
Mr. Flibble
King of the Potato People
http://www.linkedin.com/in/RobertLanning
Atom Powers
2014-05-19 22:38:58 UTC
Permalink
If I could get autofs/automount to work with CIFS that may be an option.
However it looks like credential passing could be an issue; I'll have to
look into that.

My concern is that, from reading the technet documents, Win2k8 had pretty
bad support for NFS and 2012 claims to have pretty good support. I don't
know that I trust Microsoft to go from bad to good in one release, so I'd
like to hear some personal stories about using Win2k12 as an NFS server.

We currently have a fairly old NetApp (4+ years) and want to replace it
with a SAN (maybe Nimble). We need to publish home drives and department
shares for a couple dozen Linux users on both their workstations and and
about a dozen or so shared servers; and a 10-node HPC cluster but not as
the primary/performance storage. One server should be able to handle the
load. (If it was a Linux server, and Windows can't be /that/ much worse.)
Post by Will Dennis
Looks like “CIFS” (actually SMB v2/v3) would be your best
share-publishing protocol here
 OS X and Linux have good support for this
at this point, and why make Windows Server use a not-as-well supported
protocol if you don’t have to?
If you must use NFS, looks like there is support in the product (don’t
know about 2012R2, but assume the same as 2012
) -
http://blogs.technet.com/b/filecab/archive/2012/09/14/server-for-nfs-in-windows-server-2012.aspx
W.
*On Behalf Of *Atom Powers
*Sent:* Monday, May 19, 2014 5:06 PM
*To:* LOPSA Tech
*Subject:* [lopsa-tech] MS Windows as NFS Server?
As an avenue of deprecating a NetApp device we are investigating a
block-only SAN device. That would mean using MS Windows 2012R2 to publish
shares to clients: MS Windows, Mac OSX, Linux. The MS Windows Admins manage
these file shares so Linux w/ Samba isn't an option.
Does anybody have any insight into using MS Windows as a file server for NFS shares?
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
John Stoffel
2014-05-20 01:07:15 UTC
Permalink
Atom> If I could get autofs/automount to work with CIFS that may be an
Atom> option. However it looks like credential passing could be an
Atom> issue; I'll have to look into that.

Atom> My concern is that, from reading the technet documents, Win2k8
Atom> had pretty bad support for NFS and 2012 claims to have pretty
Atom> good support. I don't know that I trust Microsoft to go from bad
Atom> to good in one release, so I'd like to hear some personal
Atom> stories about using Win2k12 as an NFS server.

Atom> We currently have a fairly old NetApp (4+ years) and want to
Atom> replace it with a SAN (maybe Nimble). We need to publish home
Atom> drives and department shares for a couple dozen Linux users on
Atom>  both their workstations and and about a dozen or so shared
Atom> servers; and a 10-node HPC cluster but not as the
Atom> primary/performance storage. One server should be able to handle
Atom> the load. (If it was a Linux server, and Windows can't be /that/
Atom> much worse.)

So why do you think Windows is the solution here? You said earlier on
that the MS Windows Admins manage the shares, but are there really
*that* many shares and do they change all that often? And why can't
the manage a simple NFS appliance or Linux server mounting the block
storage and then sharing out disk space? Throw on webmin and half the
battle is won...

I really think you're trying to fit a round peg into a square hole here.
Atom Powers
2014-05-20 01:20:06 UTC
Permalink
Because 90%+ of the users and services for that data are on Windows.
Post by John Stoffel
Atom> If I could get autofs/automount to work with CIFS that may be an
Atom> option. However it looks like credential passing could be an
Atom> issue; I'll have to look into that.
Atom> My concern is that, from reading the technet documents, Win2k8
Atom> had pretty bad support for NFS and 2012 claims to have pretty
Atom> good support. I don't know that I trust Microsoft to go from bad
Atom> to good in one release, so I'd like to hear some personal
Atom> stories about using Win2k12 as an NFS server.
Atom> We currently have a fairly old NetApp (4+ years) and want to
Atom> replace it with a SAN (maybe Nimble). We need to publish home
Atom> drives and department shares for a couple dozen Linux users on
Atom> both their workstations and and about a dozen or so shared
Atom> servers; and a 10-node HPC cluster but not as the
Atom> primary/performance storage. One server should be able to handle
Atom> the load. (If it was a Linux server, and Windows can't be /that/
Atom> much worse.)
So why do you think Windows is the solution here? You said earlier on
that the MS Windows Admins manage the shares, but are there really
*that* many shares and do they change all that often? And why can't
the manage a simple NFS appliance or Linux server mounting the block
storage and then sharing out disk space? Throw on webmin and half the
battle is won...
I really think you're trying to fit a round peg into a square hole here.
Tracy Reed
2014-05-20 06:33:19 UTC
Permalink
Post by Atom Powers
Because 90%+ of the users and services for that data are on Windows.
So we are proposing to run Windows as an NFS server and Windows as an NFS client?

"I've got a bad feeling about this..."
--
Tracy Reed
Atom Powers
2014-05-20 15:34:19 UTC
Permalink
I clarify.
200 users, 70% MS Windows, 20% Mac OSX, 10% Linux
Business data and home drives for all of the above.
Data/Shares service is "owned" by the MS Windows and Storage group(s).
Currently using mixed-mode NetApp shares: NAS.
Considering move to block storage: SAN.

Given that this is *primarily* an MS Windows service, one option being
considered is mixed-mode CIFS/NFS shares on a Windows 2k12 server.

Question: Is that a good idea? Why or why not?
Post by Tracy Reed
Post by Atom Powers
Because 90%+ of the users and services for that data are on Windows.
So we are proposing to run Windows as an NFS server and Windows as an NFS client?
"I've got a bad feeling about this..."
--
Tracy Reed
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
John Stoffel
2014-05-20 16:37:38 UTC
Permalink
Atom> I clarify.
Atom> 200 users, 70% MS Windows, 20% Mac OSX, 10% Linux
Atom> Business data and home drives for all of the above.
Atom> Data/Shares service is "owned" by the MS Windows and Storage group(s).
Atom> Currently using mixed-mode NetApp shares: NAS.
Atom> Considering move to block storage: SAN.

Atom> Given that this is *primarily* an MS Windows service, one option
Atom> being considered is mixed-mode CIFS/NFS shares on a Windows 2k12
Atom> server.

Atom> Question: Is that a good idea? Why or why not?

That makes it much more clear what you're trying to get at here. My
experience is mostly with NFS on Netapps and using Samba to share that
to Windows desktops for data interchange. We have a Windows group who
manages to the CIFS only shares on the Netapps and that works well
too.

Since you say you need some linux NFS clients, can you instead setup
the linux clients to use CIFS instead of NFS to access the data? Make
them the second class citizens in terms of authentication and ACLs and
such.

As for Windows as a file server for NFS, dunno, sorry. I like my
Netapps, they're rock solid and perform well. But they can/are
expensive to run. For your four year old Netapps, if they perform
just fine, and if you can sell it to management, why not just upgrade
them to the latest GA release, make sure it's stable then drop
support? Get some spare drives to keep on hand and go for it. Might
be quite cost effective and will let you also get the block storage
plan moving forward without having to switch all your eggs to a new
untested basket.


John
Dave Close
2014-05-21 04:44:59 UTC
Permalink
I've seen several suggestions to export CIFS shares to Linux rather than
NFS shares to Windows. My experience makes that undesireable but maybe
I've been missing something for several years.

Linux can certainly mount a CIFS share and deal with it just fine. But
every instance I've seen munges file permissions in the process. Every
CIFS file seems to have execute permission turned on, for example. File
ownership is based on who did the mount, not any attribute of the file.
To me, this makes using CIFS for anything other than access to Windows
files unacceptable. I sure don't want my home directory on CIFS.

Is there a way around these problems that I've missed?
--
Dave Close, Compata, Irvine CA +1 714 434 7359
***@compata.com ***@alumni.caltech.edu
"Political campaigns are the graveyard of real ideas and
the birthplace of empty promises." -- Teresa Heinz Kerry
Robert Hajime Lanning
2014-05-21 06:01:39 UTC
Permalink
Post by Dave Close
I've seen several suggestions to export CIFS shares to Linux rather than
NFS shares to Windows. My experience makes that undesireable but maybe
I've been missing something for several years.
Linux can certainly mount a CIFS share and deal with it just fine. But
every instance I've seen munges file permissions in the process. Every
CIFS file seems to have execute permission turned on, for example. File
ownership is based on who did the mount, not any attribute of the file.
To me, this makes using CIFS for anything other than access to Windows
files unacceptable. I sure don't want my home directory on CIFS.
Is there a way around these problems that I've missed?
CIFS associates a user with a connection. Not like NFS that associates a
user with an IO request.

NFS trusts the client to do the authentication, at least V3, I have not
looked into V4.

CIFS does a single user authentication when it receives a TCP
connection, then associates that user with all IO done via that connection.

So, not only does the security model at the filesystem level not match
between the platforms, but the user model does not either. (One user
per client for Windows and multi-user client for *nix.)

Remember what Citrix had to do it Windows NT to allow it to be
concurrent multi-user. They had to basically make it like the current
Linux Containers. All namespaces had to be sharded per connected user.
--
Mr. Flibble
King of the Potato People
http://www.linkedin.com/in/RobertLanning
Ted Cabeen
2014-05-21 22:20:42 UTC
Permalink
I think this is the best idea. If you really need support for corporate
policy reasons, there are plenty of 3rd-party companies that will
provide hardware support for EOLed NetApps for a reasonable price.

--Ted
Post by John Stoffel
As for Windows as a file server for NFS, dunno, sorry. I like my
Netapps, they're rock solid and perform well. But they can/are
expensive to run. For your four year old Netapps, if they perform
just fine, and if you can sell it to management, why not just upgrade
them to the latest GA release, make sure it's stable then drop
support? Get some spare drives to keep on hand and go for it. Might
be quite cost effective and will let you also get the block storage
plan moving forward without having to switch all your eggs to a new
untested basket.
Gary Pitman
2014-05-21 18:59:23 UTC
Permalink
Hello,

Have you considered another brand NAS? We have been running Netapps for
years and got a Isilon last Summer. I have not had any problems and it
does the mixed mode stuff pretty well so far.
Tim Kirby
2014-05-19 22:01:16 UTC
Permalink
Post by Will Dennis
As an avenue of deprecating a NetApp device we are investigating a block-only
SAN device.
That would mean using MS Windows 2012R2 to publish shares to clients: MS
Windows,
Mac OSX, Linux. The MS Windows Admins manage these file shares so Linux w/
Samba
isn't an option.
Does anybody have any insight into using MS Windows as a file server for NFS
shares?
This is something of a "how long is a piece of string" question. Do you have
any
sort of loading numbers, performance expectations, client counts etc. ? As
mentioned by Will, CIFS would probably be an easier/cleaner option... but
that depends on your performance expectations. Remember, you're replacing
a NAS, a dedicated share sharing appliance, with a server that, with all the
best will in the world, is not designed to be a file sharing appliance. CIFS
is
not generally cited as a performance protocol.

Tim
--
Tim Kirby ***@kirbys.org
Charles Polisher
2014-05-20 15:20:07 UTC
Permalink
CIFS is not generally cited as a performance protocol.
Having spent quality time with a packet sniffer in one hand and
the protocol specs in the other I can confirm that with typical
SBO usage Microsoft SMB/CIFS has terrible performance. But,
evaluate alternatives in light of training and support burdens.

HTH,
--
Charles Polisher
Will Dennis
2014-05-21 16:37:21 UTC
Permalink
Which version of SMB are we talking about here? (CIFS == pre SMB 1.0, i.e. the NT4.0 proto) MSFT is shipping SMB 3.02 now on Server 2012 R2, which is extremely high-performance (so much so that they allow SQL Server DB file access as well as Hyper-V VM file access over SMB 3.x as an alternative to iSCSI or FC connections...)

Please see http://blogs.technet.com/b/josebda/archive/2013/10/02/windows-server-2012-r2-which-version-of-the-smb-protocol-smb-1-0-smb-2-0-smb-2-1-smb-3-0-or-smb-3-02-you-are-using.aspx for further details on versions and history.

Please also take a look at http://blog.fosketts.net/2012/05/06/smb-3-huge-scope-impact/ for Stephen Foskett's take on SMB 3.0 (Stephen is a storage expert, independent blogger, and runs Storage Field Day as a part of his Tech Field Day series...)

Facts, not FUD, please.

(note that the above does not speak to the OP's original q on NFS on Win2012R2, and suggestions to consider SMB instead - just trying to make the point that SMB is a performant proto vs. NFS these days...)

- Will

-----Original Message-----
From: tech-***@lists.lopsa.org [mailto:tech-***@lists.lopsa.org] On Behalf Of Charles Polisher
Sent: Tuesday, May 20, 2014 11:20 AM
To: Tim Kirby
Cc: LOPSA Tech
Subject: Re: [lopsa-tech] MS Windows as NFS Server?
CIFS is not generally cited as a performance protocol.
Having spent quality time with a packet sniffer in one hand and the protocol specs in the other I can confirm that with typical SBO usage Microsoft SMB/CIFS has terrible performance. But, evaluate alternatives in light of training and support burdens.

HTH,
--
Charles Polisher
Charles Polisher
2014-05-23 06:13:14 UTC
Permalink
Post by Will Dennis
Which version of SMB are we talking about here? (CIFS == pre
SMB 1.0, i.e. the NT4.0 proto) MSFT is shipping SMB 3.02 now on
Server 2012 R2, which is extremely high-performance (so much so
that they allow SQL Server DB file access as well as Hyper-V VM
file access over SMB 3.x as an alternative to iSCSI or FC
connections...)
Great question. (BTW, SMB and CIFS are often used interchangably
by Microsoft. Their definitive protocol specs (MS-CIFS, MS-SMB,
and MS-SMB2) sanction interchangable usage. I haven't read MS-SMB3.)
Post by Will Dennis
Please see
http://blogs.technet.com/b/josebda/archive/2013/10/02/windows-server-2012-r2-which-version-of-the-smb-protocol-smb-1-0-smb-2-0-smb-2-1-smb-3-0-or-smb-3-02-you-are-using.aspx
for further details on versions and history.
Please also take a look at
http://blog.fosketts.net/2012/05/06/smb-3-huge-scope-impact/ for
Stephen Foskett's take on SMB 3.0 (Stephen is a storage expert,
independent blogger, and runs Storage Field Day as a part of his
Tech Field Day series...)
Facts, not FUD, please.
I'm merely reporting my own experience on a large heterogenous
net with actual servers and clients using these protocols in
production. In the right environment with the right clients and
servers, SMB 3 probably rocks. However, I was responding to a
comment (which you trimmed) about CIFS performance in general,
which I expanded on.

The Foskett article cited (I happen to be a Foskett fan) has a
lot of superlatives, was written prior to the product release,
and appears to be based entirely on product claims -- not
observed performance. He could be right, but I wouldn't call my
observations FUD, especially not while referencing Foskett's
hype!
Post by Will Dennis
(note that the above does not speak to the OP's original q on
NFS on Win2012R2, and suggestions to consider SMB instead - just
trying to make the point that SMB is a performant proto vs. NFS
these days...)
Will Dennis
2014-05-23 15:15:36 UTC
Permalink
Hi Charles,

I was just responding to your basically unilateral statement that "I can confirm that with typical SBO usage Microsoft SMB/CIFS has terrible performance." (I'm not questioning your direct experience, by the way; it would be interesting though to know when you did the testing?) That may have been the case with SMB in the past, but I know things have drastically improved from the SMB1/SMB2 days... And the OP did say he was looking at using Win Svr 2012R2 (which implements SMB 3.02.)

I too follow Foskett (more on the networking side of what he puts on with TFD than with the storage side), and I don't think he's astroturfing for MSFT... I just think he was really impressed (perhaps with pre-release data, but I haven't see anything from him since that counters this...)

It just makes me crazy when people say that MSFT tech sucks today because of their experience with it back in the NT4.0/Win 2000 Server days, and assume it's still the same way today... Technologically, they've improved their tech a whole bunch since the NT4/2K days (Windows Server, Active Directory, SQL Server, Exchange Server, scripting languages, you name it...)

(and by the way, I love, use, and appreciate *nix too; I'll use whatever the business requires to meet its goals.)

Best,
Will

-----Original Message-----
From: Charles Polisher [mailto:***@surewest.net]
Sent: Friday, May 23, 2014 2:13 AM
To: Will Dennis
Cc: Tim Kirby; LOPSA Tech
Subject: Re: [lopsa-tech] MS Windows as NFS Server?
Post by Will Dennis
Which version of SMB are we talking about here? (CIFS == pre SMB 1.0,
i.e. the NT4.0 proto) MSFT is shipping SMB 3.02 now on Server 2012 R2,
which is extremely high-performance (so much so that they allow SQL
Server DB file access as well as Hyper-V VM file access over SMB 3.x
as an alternative to iSCSI or FC
connections...)
Great question. (BTW, SMB and CIFS are often used interchangably by Microsoft. Their definitive protocol specs (MS-CIFS, MS-SMB, and MS-SMB2) sanction interchangable usage. I haven't read MS-SMB3.)
Post by Will Dennis
Please see
http://blogs.technet.com/b/josebda/archive/2013/10/02/windows-server-2
012-r2-which-version-of-the-smb-protocol-smb-1-0-smb-2-0-smb-2-1-smb-3
-0-or-smb-3-02-you-are-using.aspx for further details on versions and
history.
Please also take a look at
http://blog.fosketts.net/2012/05/06/smb-3-huge-scope-impact/ for
Stephen Foskett's take on SMB 3.0 (Stephen is a storage expert,
independent blogger, and runs Storage Field Day as a part of his Tech
Field Day series...)
Facts, not FUD, please.
I'm merely reporting my own experience on a large heterogenous net with actual servers and clients using these protocols in production. In the right environment with the right clients and servers, SMB 3 probably rocks. However, I was responding to a comment (which you trimmed) about CIFS performance in general, which I expanded on.

The Foskett article cited (I happen to be a Foskett fan) has a lot of superlatives, was written prior to the product release, and appears to be based entirely on product claims -- not observed performance. He could be right, but I wouldn't call my observations FUD, especially not while referencing Foskett's hype!
Post by Will Dennis
(note that the above does not speak to the OP's original q on NFS on
Win2012R2, and suggestions to consider SMB instead - just trying to
make the point that SMB is a performant proto vs. NFS these days...)
Ski Kacoroski
2014-05-23 15:40:29 UTC
Permalink
Will,

I agree with you on that MS tech has really improved since the NT4 days.
Powershell + Vmware + Windows 2012 is a very good combination on which
to build things. I pick the best technology for the job and in my
environment of a few hundred virtual servers supporting 3000 staff and
20000 kids, many of our most useful applications (student records,
remote file system access, website) are built on the MS stack because
of the technology integration that the vendors can take advantage of
internally in their product. As long as the vendors do not require me
to choose MS technologies for all my services (e.g. Sharepoint,
Exchange, etc.) then I am fine with it. I have not been able to make
the move to SMB2 or SMB3 yet, but it is definitely on my agenda as SMB1
is very slow.

cheers,

ski


On Fri, 23 May 2014 15:15:36 +0000
Post by Will Dennis
Hi Charles,
I was just responding to your basically unilateral statement that "I
can confirm that with typical SBO usage Microsoft SMB/CIFS has
terrible performance." (I'm not questioning your direct experience,
by the way; it would be interesting though to know when you did the
testing?) That may have been the case with SMB in the past, but I
know things have drastically improved from the SMB1/SMB2 days... And
the OP did say he was looking at using Win Svr 2012R2 (which
implements SMB 3.02.)
I too follow Foskett (more on the networking side of what he puts on
with TFD than with the storage side), and I don't think he's
astroturfing for MSFT... I just think he was really impressed
(perhaps with pre-release data, but I haven't see anything from him
since that counters this...)
It just makes me crazy when people say that MSFT tech sucks today
because of their experience with it back in the NT4.0/Win 2000 Server
days, and assume it's still the same way today... Technologically,
they've improved their tech a whole bunch since the NT4/2K days
(Windows Server, Active Directory, SQL Server, Exchange Server,
scripting languages, you name it...)
(and by the way, I love, use, and appreciate *nix too; I'll use
whatever the business requires to meet its goals.)
Best,
Will
-----Original Message-----
Sent: Friday, May 23, 2014 2:13 AM
To: Will Dennis
Cc: Tim Kirby; LOPSA Tech
Subject: Re: [lopsa-tech] MS Windows as NFS Server?
Post by Will Dennis
Which version of SMB are we talking about here? (CIFS == pre SMB
1.0, i.e. the NT4.0 proto) MSFT is shipping SMB 3.02 now on Server
2012 R2, which is extremely high-performance (so much so that they
allow SQL Server DB file access as well as Hyper-V VM file access
over SMB 3.x as an alternative to iSCSI or FC
connections...)
Great question. (BTW, SMB and CIFS are often used interchangably by
Microsoft. Their definitive protocol specs (MS-CIFS, MS-SMB, and
MS-SMB2) sanction interchangable usage. I haven't read MS-SMB3.)
Post by Will Dennis
Please see
http://blogs.technet.com/b/josebda/archive/2013/10/02/windows-server-2
012-r2-which-version-of-the-smb-protocol-smb-1-0-smb-2-0-smb-2-1-smb-3
-0-or-smb-3-02-you-are-using.aspx for further details on versions
and history.
Please also take a look at
http://blog.fosketts.net/2012/05/06/smb-3-huge-scope-impact/ for
Stephen Foskett's take on SMB 3.0 (Stephen is a storage expert,
independent blogger, and runs Storage Field Day as a part of his
Tech Field Day series...)
Facts, not FUD, please.
I'm merely reporting my own experience on a large heterogenous net
with actual servers and clients using these protocols in production.
In the right environment with the right clients and servers, SMB 3
probably rocks. However, I was responding to a comment (which you
trimmed) about CIFS performance in general, which I expanded on.
The Foskett article cited (I happen to be a Foskett fan) has a lot of
superlatives, was written prior to the product release, and appears
to be based entirely on product claims -- not observed performance.
He could be right, but I wouldn't call my observations FUD,
especially not while referencing Foskett's hype!
Post by Will Dennis
(note that the above does not speak to the OP's original q on NFS
on Win2012R2, and suggestions to consider SMB instead - just trying
to make the point that SMB is a performant proto vs. NFS these
days...)
_______________________________________________
Tech mailing list
https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
--
"When we try to pick out anything by itself, we find it
connected to the entire universe" John Muir

Chris "Ski" Kacoroski, ***@gmail.com, 206-501-9803
or ski98033 on most IM services
Continue reading on narkive:
Loading...