Results 1 to 7 of 7
  1. #1
    Join Date
    Sep 2002
    Location
    back of some datacenter
    Posts
    140

    Problem with customer over Windows 2000 server install (your opinion please)

    Hey guys I’m looking for people with Windows 2000 experience or anyone in general that understands this setup to comment on this post, as I’m going to direct my customer to this post if they continue to not take my word for things.

    Several months ago I was contracted to install Windows 2000 server at a location, and configure the 7 Windows XP workstations to logon to it. OK sounds routine enough so I go and look over the job.

    They had Cat-5 cable installed when the place was built; it all came into the wiring closet neatly.
    They have a Cisco 1700 series router in there with a T1 WIC card, the data side of the T1 is 12 channels the other 12 are split off to the phone network for voice.
    The odd part was when I checked a few of the workstations they were using public IPs, so whoever setup the network to begin with opened it up to the world and all the workstations have been sitting there ever since public as can be. So I said ok no problem lets get the server going and I’ll come up with some options for fixing that later.
    I then setup Windows 2000 server as an active directory domain controller and all that good stuff, setup the users, shares, and printers, then configured the workstations to logon to it. I’ve done so many setups like this I can nearly do it blindfolded, so that was more or less it I did some testing to make sure everything worked, I then went over a few basic things like how to share a folder and how to add a user general day to day stuff.

    That was it till around the first of January when they call about it not working, now there’s no way it should have failed that soon. I started troubleshooting in my head based on what they told me over the phone, and somehow this thing had got itself into setup mode after a restart, I give up on fixing with support over the phone and told them I’d come look at it.

    So here’s where the fun starts I get down there and the server is sitting on the setup screen waiting for a key code, keep in mind that they have data on the drives that I have to recover. I shut down the server and installed a 80GB hard drive, I did this because they had two 60GB drives striped on RAID 0 so I had to install Windows 2000 server on the 80GB drive and setup the RAID controller to copy the data over to the 80GB. After all the data was recovered I setup the RAID controller in the BIOS as the primary drives again, then I went to formatting and installing Windows 2000 server.

    During the time of troubleshooting I was told by a person there that one person installed a piece of software and the server started acting funny after that, if I remember correctly they tried to uninstall it and stopped it halfway though so it started acting up even more. I was also told that someone else turned some “stuff” on and off, so from what I gathered at least three different people did something to the server. This is why I chose to just format and reinstall since I managed to recover all the data. Anyone with Windows 2000 server experience knows there are A LOT of things you can turn on and off in the administrative tools panel, so the first day I was working on this from about 6PM to 12AM most of that time being installing, recovering, and re-installing Windows.

    The next day I got there Windows was crashing on me from time to time and sometimes even blue screening, so I shut it down again popped the 80GB drive back in and booted off it to format the RAID drives and run scan disk with surface check. Everything checked out so I re-installed Windows again on the RAID drives, it was working fine this time so I start setting up active directory, users, shares, etc…

    After everything was all setup on the server I went around to all the workstations to check them since the profiles can act funny after a server re-install, so I got them checked out then made sure all the logons and shares worked ok. That was more or less it, I told the person there that does computer stuff that he needed to change the password to something other than “test” I use simple words during setup and configuring so I don’t have to type something like “G5u6fY3t9” during all the setup re-boots. That was it I was finished.

    About 2 weeks later I logged into the server via terminal to make sure Norton Antivirus was updating like it should and running the full scans at 3AM every morning. I didn’t think I’d get in without calling them for the new password but I had nothing else to do so I logged in to the administrator account and tried the password “TEST” about 2 weeks later and it was still the same.

    While trying to login I noticed that it was running really really slow and this is at 11PM, keep in mind that they have a Fractional T1 at 768k so that gives about 80KBps, my first thought was someone was downloading a huge file overnight. So me being the picky type to see what’s going on, I ran netstat on the server and saw a lot of traffic on port 6667 yep you guessed it IRC, I then turned on SNMP and told one of my MRTG servers to watch it…for about an hour I watched it kick out a steady 68KBps…

    During this time I went on to check Norton Antivirus and do a full scan, it hit on a IRC Trojan so I went to work digging though the server removing all traces of it, and scanning several more times to make sure. Now since they don’t check email on this server then the only quick and easy way anyone could have got in was using the password “test” now in most cases that wasn’t a smart idea but after all the calls from customers about the passwords that look like “Hs3FkyT63” I leave them with, I try to make it simple for them if they have someone there that can handle computer stuff for them.

    They told me they had someone there that could handle running the server so I said ok, most of the time my clients buy a support/management contract from me after the initial setup and they never have to touch their server(s) again with the workstations being covered as well, as it’s all handled from my end I sometimes have to ask them to put a CD in the server CD-Rom but other than that I handle everything right down to the passwords. I don’t mind if a customer wants to manage things themselves but that’s assuming that they have some knowledge in the area.

    Anyway I fixed the Trojan problem, changed the administrator password to something hard, and kept the server on MRTG for several days to make sure that I did fix the problem. Ever since then MRTG has shown 0% after hour’s traffic, the logs are showing nothing other than general server info and everything seems to be fine.

    So after all of that I faxed them some general documentation on the network layout, Visio charts and such, when I first setup the server they couldn’t find how many IPs they had or what was assigned to what so I went the extra step and I mapped the network and charted it for them to have a copy, showing IPs, system names, and how they were connected to the network I also sent them a layout for a future firewall setup so they could look it over. I was late mailing the bill for the last server re-setup and faxing the info since my dad was in the hospital, I took care of the IRC Trojan for free since it was an immediate problem and I didn’t want to re-setup the server again after some kid with a password starts messing stuff up.

    I got another call the day before the bill was due which was Monday, I was told that they had someone come in and look at they’re systems and that person told them that they are wide open to the internet, I was telling them that from day one. I was then told that it wasn’t like this before I setup the server, but it was so whoever originally set it up or told them misinformed them of what they had because there was NO firewall in place when I first walked in the door and all the computers on the network were and still are wide open to the world. The router is plugged straight into one of the switches an there is nothing between them, the firewall suggestion I faxed them was a layout for a simple cheap Unix/Linux based firewall with IP Tables to simply drop all incoming packets if they wanted to keep using the public IPs or I could switch it to NAT mode and give them 254 internal IPs vs. the 15 public IPs they have now.

    As far as my Microsoft background I’ve been a MCSE since 99 for NT and Windows 2000 since 00, I currently have setup in the past year 42 Windows 2000 networks ranging from 4 workstations and a small server up to 83 workstations and 14 servers in a Windows/Unix/Linux mixed environment. I currently hold support and management contracts with about half of those companies.

    This current job is the only one out of the 42 jobs I sold last year that I’m having a problem with, so feel free to comment on this as I decided to post this publicly for a second opinion and stake my reputation on it that I’m not in the wrong here. I have server management contracts with some of you so I’m risking those if I’ve done something wrong in the above information by making it public.

    Sorry for the long post but I have not had any contact with them since yesterday over this matter and as most of you know it’s very hard to explain technical stuff to non-technical people, I just wanted to make sure I had some second opinions to direct them to.

    Thanks for your time,

    Jason

  2. #2
    Join Date
    Jan 2003
    Location
    Cardiff, UK
    Posts
    128
    You are 100% correct in your actions and information past to your customer. Once you have left the building and informed the customer that its now there responsibility to administer the systems, due to the fact there did not take out a contract for you to administer the systems.

    If they called in another company to have a look at the stystem, then so be it. But did you state in writing right from the begining that the whole network was open to possible outside abuse........ AND it had NO firewall/protection installed. If you did give this in writing, then you are covered.

    All I would do is say thay you are willing to correct the problems, providing they take out a contract and let you soley administer the systems. This way they know who has full control over the system.

    It would leave the ball in there court and if they decide not to take out a contract, then you have done all they you were paid to do and more for that matter.

    I had a similar problem with a customer with 1 network server, 1 printer server and 68 work stations. They tried to pass the blame on to me but I always inform customers in writing and everytime I'm on customers promises, if I see anything that Im not happy with then I inform them and write it all down on work sheets and get the customer to sign it.

    This is usefull to use as evidence to prove to a customer that you pointed out a specific problem to them weeks ago, and they choose not to have the work done.

    Hope it all works out well for you.

    Regards,

    Brian

    A+ N+ MCSA MCSE

  3. #3
    Join Date
    Oct 2002
    Location
    Canada
    Posts
    3,103
    Unless you were initialy contracted to setup the firewall as well, i'd say send the bill for troyan removal and drop the client.
    cna, a+, mcse2k

  4. #4
    Join Date
    Sep 2002
    Location
    back of some datacenter
    Posts
    140
    Well that's the way I looked at it, all I was there to do was install Windows 2000 Server and configure the workstations to logon to it. That had nothing to do with the network layout, they told me that they were using public IP's after I already figured it out for myself and they had some kind of security camera software that ran on one of the computers so you could logon from the internet using that computer’s IP address and view the cameras. It was at this point that I assumed they wanted the network left the way it was, then after that little Trojan issue I sent the layout for a suggested firewall. But now they clam that they had one so someone down the line told them wrong or they just assumed they were secure all along.
    Sitestash, LLC.
    Shared / Reseller / Dedicated / Colo
    877.697.7286

  5. #5
    Join Date
    Sep 2002
    Location
    Knoxville, TN
    Posts
    645
    Do you really need this contract? Is it cost effective for you to continue to work with this company?
    I think those are important questions at this time.
    It looks like you're spending a lot of time on it that you're not billing them for and yet they don't have confidence in what you do, which is why they called in someone else or listened to someone else who is trying to sell them a service (the second option would be the most likely).
    It sounds like they're so clueless that they don't understand what you're doing for them and never will appreciate it.
    If I were in your shoes, I'd take the initiative, suggest they'd be better served by another company and walk away.
    Spend your time and energy on finding a better company to work for.
    Laura K.
    http://www.madmousergraphics.com
    graphic design for grownups

  6. #6
    Join Date
    Sep 2002
    Location
    back of some datacenter
    Posts
    140
    I just spent about an hour on the phone with them, the person that looked at they're server told them they were wide open to the world and he got in. Now they said this person gave them several sheets showing what’s open and such, my guess is he port scanned the IP block and it showed ports open he probably threw in some technical stuff to fill up the pages.

    The main problem they had with me was that no firewall was installed, so I tried my best to explain how IPSec works and that it was setup, but when you have one person saying a port is open and that's bad, then another person saying it's supposed to be open it tends to confuse people.

    I guess the other guy was trying to make some easy money, he clamed to have gotten into the server but they didn't have the papers in front of them to tell me how he did it. Plus they told me HE CAME IN AND LOOKED at the systems; well wouldn't they have to give him the ADMIN PASSWORD for him to check out the server? Well duh, that makes it real simple to get in from the outside now...when someone is scanning for security holes in a system the LAST thing they need is the admin password. So I'm betting he scanned the system for open ports, logged in using the admin password, and told them they were "open to the world, give me money and I'll fix it". They also said they were getting messages that popped up saying “this computer is not secure” and other such messages, they said this wasn’t happening before…does messenger service spam ring a bell? I know most of you will agree with me that it’s quite simple to make a believable message pop up on any Windows NT/2000/XP computer that has that service running. I didn’t even try to explain that as it would have looked like a security mistake on my part when in fact it’s quite harmless and easy to turn off. My guess to that is some company is offering security services by scarring the customer into believing that they got into the system and left a message saying the system is unsecured.

    So now I'm the "bad guy" since they are "open to the world" according to some guy that's trying to make a buck and make me look bad in the process.

    One of them went out and bought a cable/DSL router apparently trying a "do it yourself" version of the suggested firewall layout I sent them, I asked if it supported NAT and it did so I offered to set it up it myself so everyone’s happy, I can be done with this and get paid for the last invoice I sent them. So while trying to set it up over the phone the only IP and subnet info I had was what I mapped out myself, now I have to wait for them to call XO and get the IP information that's assigned to them so I can finish setting up the router behind the current Cisco router. I was going to just configure the Cisco router to do NAT but no one knows the password for it or has it written down somewhere and telnet is turned off on it for some reason, so I'm down to the option of piggy-backing the cable/DSL router onto the Cisco router to do NAT for the network. Now there are going to have a crazy little routing setup, but at least it will work and no one can scan the network claming that they are open to the world and I didn’t do my job correctly.

  7. #7
    Join Date
    Jan 2003
    Location
    Cardiff, UK
    Posts
    128
    Great...... glad it all worked out in the end. I would get the last invoice out of them, do this little job and drop them once and for all.

    Saves you a load of hassle at the end of the day.

    Regards,

    Brian

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •