summaryrefslogtreecommitdiff
path: root/content/blog/mail-disaster.md
blob: 86706631e82c5940100c7d806a88c14aa132b6a2 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
+++
title = "Mail Problems"
categories = [ "Mail", "Linux", "Security" ]
date = "2019-03-29T12:58:31+01:00"
thumbnail = "/images/blog/mail-disaster/mail-disaster.png"
+++

## History

It was a beautiful day. My mailserver on the Raspberry Pi B was running
without any issues for some time now.

In the evening of March 12th I got a nice email from my external DNS
provider:

```
The BuddyNS janitor writing. A safety notification on your BuddyNS account:

    Your zones reached 60% of your account's traffic quota.

Details:
* Total traffic produced this month: 181 Thousand queries.
* Current traffic quota: 0.3 Million queries/month.
```

Well, fine, I thought, finally somebody is checking on my web page and I
went to sleep.

Of course this was not the case: I had a weak password in one of the accounts
of my mailserver (which allowed any legitimate Linux user to send
emails). This caused all those DNS lookups for my domain on the
BuddyNS DNS servers.

So, my thinking went along the lines: well, some weeks ago
I replaced the SD card, because the old one was worn out, I cannot
remember whether I replaced all standard passwords. My suspicion got
confirmed when I saw the following line in the my mail log:

```
From: "George"<alarm@andreasbaumann.cc>
```

Swearing big times about my own stupidity (the default password for the
'alarm' account is - well - weak) I started cleaning up the mess.

Checking my mail server logs I found that all attacks went via one single
IP (185.228.80.18). So just blocking the firewall was the fastest way to
fix the tousands of spam email being sent via my now defacto open mail relay.

## Checking status

There are various helpfull tools to check the status of your mail
server. I picked https://mxtoolbox.com/. This is what I got:

```
 dmarc  andreasbaumann.cc  DNS Record not found   
 blacklist  smtp.andreasbaumann.cc  127.0.0.2   
 blacklist  smtp.andreasbaumann.cc  Blacklisted by JUNKEMAIL
 blacklist  smtp.andreasbaumann.cc  Blacklisted by NIXSPAM   
 blacklist  smtp.andreasbaumann.cc  Blacklisted by TRUNCATE   
 blacklist  smtp.andreasbaumann.cc  Blacklisted by UCEPROTECTL1   
 blacklist  smtp.andreasbaumann.cc  Blacklisted by WPBL   
 mx  andreasbaumann.cc  No DMARC Record found
 mx  andreasbaumann.cc  DMARC Quarantine/Reject policy not enabled   
```

I also like the results from http://zy0.de/q/83.150.2.48:

{{< figure src="/images/blog/mail-disaster/zy0_de.png" alt="zy0_de check resulst for 83.150.2.48" >}}

Especially it shows you headers of SPAM mails, which are quite helpful
to detect, what went wrong:

```
Spam samples A small selection

 12.03.2019 02:03 (Z) (date of processing)

Return-Path: <alarm@andreasbaumann.cc>
X-Original-To: cindy@SPAMTRAP.INVALID
Received: from smtp.andreasbaumann.cc (smtp.andreasbaumann.cc [83.150.2.48])
 by mail.ixlab.de (Spamtrap) with ESMTP
 for <cindy@SPAMTRAP.INVALID>; Tue, 12 Mar 2019 03:03:20 +0100 (CET)
Received: from User (unknown [185.228.80.18])
 by smtp.andreasbaumann.cc (Postfix) with ESMTPA id 909CD77F2A;
 Tue, 12 Mar 2019 01:22:20 +0100 (CET)
Reply-To: <gg828579@gmail.com>
From: "George"<alarm@andreasbaumann.cc>
Subject: Good Day!!
Date: Mon, 11 Mar 2019 17:22:27 -0700
MIME-Version: 1.0
Content-Type: text/html;
 charset="Windows-1251"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
X-NiX-Spam-Hash2: 5994f93f698c55d5b527b1da55f31611
X-NiX-Spam-Source-IP: 83.150.2.48
X-NiX-Spam-MX: mail.ixlab.de
X-NiX-Spam-Listed: yes
```

## Blacklisting

Mail servers can ask blacklists for bad IPs or domains and then block
incoming mails.

Most blacklists give you a home page, where they explain, how they 
manage the list. There you might also find the status of your IP or domain.

There are basically three ways you can try to get off such a list:

* you can fill in a form, usually describing what went wrong and how
  you solved the problem.
* you have to send an email with basically the same kind of information
* you can do nothing, the delisting happens automatically

Keep in mind, that humans read those messages, be polite and be open
about what went wrong. I never had a problem getting delisted, when
I described, what I did wrong in the past and how I will enforce better
security in the future.

Also note: you usually don't get any email or feedback. Give people time
and they will consider the case. If they think, you deserve to send
emails again, they will delist you from the blacklist.

Find below short descriptions of what I had to do in the individual cases.

### JumkMailFilter

Visited the "remove from the list" for at:

https://ipadmin.junkemailfilter.com/remove.php

Entered my IP and some text, why I got onto the list.

### DNSBL

http://www.dnsbl.manitu.net/remove.php?value=83.150.2.48

I had to fill in a form and describe, what went wrong on my side and
how I fixed the problem.

### TRUNCATE

http://www.gbudb.com/truncate/index.jsp

Had nothing to do here, but wait:

```
"Maintenance of this list is completely automated and there are no
provisions for the manual addition or removal of entries."
```

### UCEPROTECTL1

```
"This blacklist does not offer any form of manual request to delist. 
Your IP Address will either automatically expire from listing after
a given timeframe, or after time expires from the last receipt of
spam into their spamtraps from your IP Address.

There is an express delisting for 89 CHF
```

For a personal domain I can wait for seven days sending out no spam.

For a business domain I would most likely pay the 89 CHF. :-)

### WPBL

http://www.wpbl.info/

```
IP addresses are automatically removed with time, after
spam stops arriving. For example, a lone spam sighting
will only get an IP listed for 7 days. You can also
remove an IP address using the Lookup facility at the 
top of the page. This no-questions-asked, instant removal
facility is provided for the benefit of administrators
who feel that the record is in error or have fixed the
security problem that allowed spam to be sent through
their hosts. Access to the removal facility may be
restricted if there is any abuse of our system, including
attempts to automate removal of multiple IPs using
scripts. Removed records still remain in database backups. 
```

Clicking on:

http://www.wpbl.info/cgi-bin/remove.cgi

I got:

```
Found IP address 83.150.2.48 in database, marking for removal.
Record removed. The published list is updated hourly, so changes may not show immediately. 
```

### SPAMCOP

https://www.spamcop.net/w3m?action=checkblock&ip=83.150.2.48

I filled in the provided form.

### IBM DNS

This is a nice security product called 'IBM-X-Forge-Exchange', 
so I had to log in with my IBM Id.

{{< figure src="/images/blog/mail-disaster/ibm.png" alt="entries in IBM-X-Forge-Exchange" >}}

I also had to describe my case to get delisted.

### Gmail

Now this one was tricky. Google has a not-so-great postmaster tool, hard
to find forms to fill in and some confusing documentation.

I tried here:

https://glockapps.com/blog/remove-ip-address-gmail-blacklist/

https://support.google.com/mail/contact/msgdelivery

The postmaster tools are not a big help, really, I registered nonetheless.

I got reject till March 28th, as far as I can tell the domain reputation below
is one of the worst ones you can get and the only option is to wait some weeks
after filling in the forms:

```
Our system has detected that this message
is 550-5.7.1 likely suspicious due to the very low
reputation of the sending 550-5.7.1 domain
```

## Course of Action for a better mail service

I made sure, I have some security standards in place, so that
at least faking the domain in the 'From:' field is not so simple:

* [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework): Sender Policy Framework
* [DKIM](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail): Domain Keys Identified Mail
* [DMARC](https://en.wikipedia.org/wiki/DMARC): Domain-based Message Authentication, Reporting and Conformance

Those things don't help against a broken account on the mail server,
as in my case, but they provide positive rating for emails being
judged in the future, and they are simple to implement.

I also added a list of accounts/emails to the postfix configuration.
Only those accounts are allowed to send emails from the host.
Even if this means you have to generate the entry in '/etc/passwd'
and another one in that postfix list. This makes sure,
no "rogue" Linux account can be abused for sending emails, when
compromised.

I added myself to the [DNSWL](https://www.dnswl.org) white list too.

And of course, I deleted the 'alarm' account on the machine. :-)

### Update 4.4.2019

Gmail is still blocking me (or again?). So is bluewin.ch. The mess is
not over, so I can only recommend everybody to make sure not to get
into this situation in the first place.

Added [Fail2Ban](https://www.fail2ban.org) to filter for common Postfix
and Postfix SASL errors, like password-breach attempts via SASL. This
works like a charm.

Added [Spamassassin](https://spamassassin.apache.org/) and
[Razor](http://razor.sourceforge.net/) to get rid of spam.

### Update 15.4.2019:

Gmail likes us again.. :-)

## References

* https://mxtoolbox.com/
* http://zy0.de/
* https://en.wikipedia.org/wiki/Sender_Policy_Framework
* https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail
* https://en.wikipedia.org/wiki/DMARC