Problems with email?
10-06-2005, 01:32 PM
#21
Erik
Yes, I know you are working on it. But can you explain this section of the header.
The way I read this, Shark.s2ki.com passed the message to Shark.s2ki.com on Mon, 03 Oct 2005.
And it was submitted to shark.s2ki.com on Thu, 15 Sep 2005.
So why is there an issue with SMTP passing the mail from Shark to Shark? That does not make sense. Unless the IMAP web client in this case, is submitting the message but never closing the message (EOM) and it is hung at that point. Then something on the 3rd caused these hung messages to get released.
The headers looked the same for the PM notifications too. To the letter.
Yes, I know you are working on it. But can you explain this section of the header.
Mon, 03 Oct 2005 21:21:32 -0700
Received: from shark.s2ki.com (localhost.localdomain [127.0.0.1])
by shark.s2ki.com (8.13.4/8.13.4) with ESMTP id j8FKk4Iq001896
for <Jeff-Ricks2k.com>;
Received: from shark.s2ki.com (localhost.localdomain [127.0.0.1])
by shark.s2ki.com (8.13.4/8.13.4) with ESMTP id j8FKk4Iq001896
for <Jeff-Ricks2k.com>;
And it was submitted to shark.s2ki.com on Thu, 15 Sep 2005.
Thu, 15 Sep 2005 13:46:04 -0700
Received: (from httpd@localhost)
by shark.s2ki.com (8.13.4/8.13.4/Submit) id j8FKk4nB001895
Received: (from httpd@localhost)
by shark.s2ki.com (8.13.4/8.13.4/Submit) id j8FKk4nB001895
The headers looked the same for the PM notifications too. To the letter.
10-06-2005, 01:43 PM
#22
Registered User
Thread Starter
Originally Posted by HondaGal,Oct 6 2005, 01:23 PM
37 minutes between posts? common. I assure you email is a priority cthree will be working on it.
I do know people are working on it....
10-06-2005, 03:02 PM
#23
Administrator
Jeff,
That's actually an entirely different email problem (there are plenty to go around). The issue with webmail is related to the IMAP server process not properly authenticating people (it says LOGIN FAILED in the logs) when they login to collect their mail from their s2ki.com mailboxes. Webmail logs in via the IMAP server therefore it doesn't work either.
The problem you are looking at is an issue with the SMTP configuration on shark which is one of the web servers in our cluster. shark doesn't receive mail of any kind. It takes the messages generated by the web server scripts and delivers them to the destination using SMTP. What it looks like from the information you posted is that shark's mail settings are mis-configured and that it's looping mail back to itself rather than sending it.
The way things are configured shark should be sending all of it's outbound mail to a different server, our mail server, to be relayed on to you. That way all of the mail generated by the web servers appears to come from just one server, s2ki.com. It also provides that there is only one server that needs to deal with the complications of queuing, scheduling and delivering mail messages rather than configuring every web server to do this itself. The only time mail would be queued or delayed on the web server itself is if it couldn't contact the mail server to unload it.
I need to double check that shark (and the other web servers) are configured correctly as described above. There are a number of reasons why shark might have stopped delivering mail. Monitoring the mail on the webservers is not something I normally do. They are configured to be as lightweight as possible and so I just assume they are going to work. I've added closer monitoring of the state of mail on the web servers to the server management system I use so hopefully in the future I'll be able to detect a problem much more quickly in the future.
Most of the problems that occur you never know about because I have tools I use to detect them early and fix them before you even know they happened. We have redundancy built into most of the systems as best I can afford but there are still a few weak points that can take down the whole works. The firewall is one example of something I can't easily or affordably make redundant so if it crashes or dies we're screwed. Mail is another soft spot. It's just not something I can economically make fault tolerant.
Some examples of redundancy are for example:
- multiple load balanced web servers which can handle the system load without interruption even if one crashes or dies.
- multiple database servers synchronized in near real time each with their own fault-tolerant RAID disk arrays. A failure of the primary database server only requires a simple configuration change (change the name of the database server) to switch over to the stand-by server. We run searches on the stand-by server to give it something to do while it's waiting for the primary to die.
- redundant disk arrays for the photo gallery and all of the avatar images and such shared by the web servers
- redundant disk arrays for storing email
There are lots of other things I'd like to add but our rackspace, power availability and budget is only so big. One thing people don't often consider is that servers need to be plugged in. We only have 15A available for our server rack and there are over a dozen computers plugged into that. I'm not sure if I could add another server right now or not without blowing the breaker even if there was room to cram it in. The cost of adding another server might easily double our already staggering hosting costs if I can't power it up and need to get another rack.
Anyway I'm still looking into these mail issues to both fix them and to try and prevent them from happening again.
That's actually an entirely different email problem (there are plenty to go around). The issue with webmail is related to the IMAP server process not properly authenticating people (it says LOGIN FAILED in the logs) when they login to collect their mail from their s2ki.com mailboxes. Webmail logs in via the IMAP server therefore it doesn't work either.
The problem you are looking at is an issue with the SMTP configuration on shark which is one of the web servers in our cluster. shark doesn't receive mail of any kind. It takes the messages generated by the web server scripts and delivers them to the destination using SMTP. What it looks like from the information you posted is that shark's mail settings are mis-configured and that it's looping mail back to itself rather than sending it.
The way things are configured shark should be sending all of it's outbound mail to a different server, our mail server, to be relayed on to you. That way all of the mail generated by the web servers appears to come from just one server, s2ki.com. It also provides that there is only one server that needs to deal with the complications of queuing, scheduling and delivering mail messages rather than configuring every web server to do this itself. The only time mail would be queued or delayed on the web server itself is if it couldn't contact the mail server to unload it.
I need to double check that shark (and the other web servers) are configured correctly as described above. There are a number of reasons why shark might have stopped delivering mail. Monitoring the mail on the webservers is not something I normally do. They are configured to be as lightweight as possible and so I just assume they are going to work. I've added closer monitoring of the state of mail on the web servers to the server management system I use so hopefully in the future I'll be able to detect a problem much more quickly in the future.
Most of the problems that occur you never know about because I have tools I use to detect them early and fix them before you even know they happened. We have redundancy built into most of the systems as best I can afford but there are still a few weak points that can take down the whole works. The firewall is one example of something I can't easily or affordably make redundant so if it crashes or dies we're screwed. Mail is another soft spot. It's just not something I can economically make fault tolerant.
Some examples of redundancy are for example:
- multiple load balanced web servers which can handle the system load without interruption even if one crashes or dies.
- multiple database servers synchronized in near real time each with their own fault-tolerant RAID disk arrays. A failure of the primary database server only requires a simple configuration change (change the name of the database server) to switch over to the stand-by server. We run searches on the stand-by server to give it something to do while it's waiting for the primary to die.
- redundant disk arrays for the photo gallery and all of the avatar images and such shared by the web servers
- redundant disk arrays for storing email
There are lots of other things I'd like to add but our rackspace, power availability and budget is only so big. One thing people don't often consider is that servers need to be plugged in. We only have 15A available for our server rack and there are over a dozen computers plugged into that. I'm not sure if I could add another server right now or not without blowing the breaker even if there was room to cram it in. The cost of adding another server might easily double our already staggering hosting costs if I can't power it up and need to get another rack.
Anyway I'm still looking into these mail issues to both fix them and to try and prevent them from happening again.
10-09-2005, 09:07 PM
#26
Administrator
I completely reconfigured the email this weekend and put the Webmail system back online. It works for me, let me know if it's not working for you.
Part of the configuration was an overhaul of the SPAM filters. They should be fully up to date and so far they have been 100% accurate for me. Time will tell...
Part of the configuration was an overhaul of the SPAM filters. They should be fully up to date and so far they have been 100% accurate for me. Time will tell...
10-10-2005, 09:32 AM
#27
Administrator
Erik, what do the 
filters actually do? do they physically stop the email, or are they what marks the email with [SPAM] in the subject field? I got a few with [SPAM] in them today.
I've not had any probelms with the email btw, but i use POP all the time, no webmail. Apart from the laxative that you gave the webserver which spat out a couple of old post reports
filters actually do? do they physically stop the email, or are they what marks the email with [SPAM] in the subject field? I got a few with [SPAM] in them today.I've not had any probelms with the email btw, but i use POP all the time, no webmail. Apart from the laxative that you gave the webserver which spat out a couple of old post reports
Thread
Thread Starter
Forum
Replies
Last Post