e164.org Service Update/Planned Outage

evilbunny

Neuer User
Mitglied seit
1 Mrz 2006
Beiträge
132
Punkte für Reaktionen
0
Punkte
0
Sent to the e164.org mailing list on the 16th of September

----

This week we are moving the main website to a new location, the move is
because the previous hosting sponsor is no longer going to be in
business. e164.org gratefully appreciates all hosting and support we
received in the past.

We are also going to be bringing a UK secondary server online shortly
after we finalise any move to increase global diversity, and are
pondering over options for other continents. For those in the UK/Ireland
and most of western Europe this should decrease lookup times.

Just to recap, some time this week, will post another note 24 hours
prior, we will be shutting down the current web site/server and doing a
final sync with our new hosting then bringing the new site online and
updating DNS records etc.

--

Best regards, Duane
 
posted to the e164.org mailing list on the 25th of September

---

We were hoping to have things transferred by now, but it hasn't
happened, I'm going to be spending most of tomorrow (east coast Aussie
time) finalising and making sure everything is in place and then the
change over will occur at some point after that.

In other news we now have a UK secondary name server and I'm also just
finalised the setup of a server in Australia.

We're also considering grouping our name servers rather then our current
situation. The idea behind this is two fold, firstly if you have too
many NS records listed in a zone it can cause all DNS requests to be
sent via TCP, which are dramatically slower then a single UDP packet.

This would also give us the ability to put name servers into "zones"
this way we can have;

* na.e164.org point to all North American name servers (3+ at present)
* sa.e164.org point to all South American name servers (none at present)
* eu.e164.org point to all European name servers (1 at present)
* pa.e164.org point to all Pacific nations name servers (au/nz/etc - 1)
* as.e164.org point to all Asian name servers (none at present)
* af.e164.org point to all African name servers (none at present)

>From here people can add a small zone forwarding stub to BIND, PDNS and
some other recursive name servers that have this option. It's a kind of
poor mans anycast, we'd really like to do anycasting, but the cost far
out weigh any benefits, compared to a small amount of time to change
your config file to optimise lookup speeds for your part of the globe.

Bind Example:

zone "e164.org" {
type forward; forward first;
forwarders {
<ip.of.closest.nameserver>;
<ip.of.2ndclosest.nameserver>; # optional
};
};

PowerDNS Example:

forward-zones= e164.org=<ip.of.closest.nameserver>

After going through and manually ordering the list in bind for testing I
think it might be a good idea to come up with a simple shell script to
lookup the current domain servers and then output the optimal list to a
config file and restart bind.

Alternatively the developer of PowerDNS suggested BIND and PDNS in
recursive name server operation will initially send requests to
different zone name servers until it exhausts the list and continues
using the most responsive.

One final point, after my last email announcement I received a number of
offers for hosting our zone, for which we are very grateful for the
offers, however I've finally gotten round to putting our current
criteria for consideration to our wiki:

http://www.e164.org/wiki/Infrastructure

More to come shortly.

--

Best regards, Duane
 
Sent to the e164.org mailing list on the 29th of September

-----

I now have a working replicate of the main webserver and will begin shutting down services on the existing server so they can be transferred to the new one.

--

Best regards,
Duane
 
Sent to the e164.org mailing list on the 29th of September

-----

I've just finished fixing/changing/etc everything that needed to be
fixed or changed and hopefully everything is working.

If DNS hasn't updated for you yet and you want to help test things
out/report problems do the following:

echo "208.70.151.161 e164.org www.e164.org forum.e164.org wap.e164.org" >> /etc/hosts

You will need to remove this in a day or so, otherwise you will most
likely have problems if there are other changes made in future.

--

Best regards,
Duane
 
Of late I've been making some fairly wide ranging changes to e164.org and this has caused some code to break and I'm sorry for any inconvience this may have caused anyone.

That said the changes I've been making are supposed to make things better in the long term, I've been making changes to the website to greatly improve distribution of the zone data.

In the begining we were using zone transfers, but this is inefficient and after a while starts churning through CPU and bandwidth.

Then we switched to bind style zone files on HDD and distributing this via rsync over SSL, this too had it's issues with getting the file to the hdd in the first place and/or getting daemons to re-read the files after changes occur and after all that manipulating the flat files on the hdd in an efficient manner.

MySQL replication, and any other SQL/LDAP type backends incur a fair bit of performance penalties, like most other things when you start dealing with really really big zone files, it's not as big as most/some of the ccTLDs, or the gTLDs, but we're getting close to 19M of data in bind style zone files, because these were pre-loaded into memory the DNS servers run really quickly, but take a big hit when the zone file has to be re-read.

So I started hacking some DNS frame works to try and make somethig that suits our needs, I was considering doing our own hash tables, but decided to use memcached once I had a basic but working daemon.

However memcache isn't really designed for this purpose, and it lacks functionality to save to the hdd and to read data from the hdd into memory so I went looking at Tugela cache which was a forked version of memcache however this isn't actively developed so I was wary of going down this path.

So anyways, Tugela actually ripped out a bunch of code and added BDB code to store things that wouldn't all fit in memory to HDD and so I looked futher into this option, and PHP on Ubuntu/Debian was packaged with native support for DB4 support.

The funny thing about utilising a DB4 file, it actually sped up the rate at which information could be returned through the DNS frame work, at a best guess the latency was due to TCP overhead, though I've no idea how to prove that, and I'm guessing the DB4 file is sitting in RAM cache as well, so that would speed things up as well, the 2 storage methods were very similar, so it was pretty trivial to switch from using memcached to DB4.

The only down side is PHP code running on the website can't easily access manipulate data in the DB4 file and be secure on the server, but I got round this by sending update notification from the website in a UDP notify packet and then the DNS server runs update code via an SSL connection grabbing a list of incremental updates and processing them.

This is where website changes come into play, numerous sections of the website had to be altered to capture and store all adds, updates and deletes over a 48 hour period, anything out of sync more then that can just do a complete copy of the data, I haven't really worked that side of things out, but once I'm sure everything implemented so far is working properly I'll extend things further.

One of the features I've wanted for a while it to be able to cope with virtually any TLD or ccTLD. We currently have 5 domains, 3 ccTLDs and 2 gTLDs, which we're planning to setup in a fairly redundent manner once everything else gets sorted out.

The domains we have are:

* e164.org
* e164.tv
* e164.biz
* e164.ws
* e164.org.au

The system can cope with virtually with any 2nd or 3rd level domain with 'e164' in it, including e164.arpa, so if anyone has an e164.arpa dellegation or looking to get it delegated for your region/country and you have no idea what to do with it, you can soon get the NS records set to our name servers and it will just work.

We are more then happy to discuss anything from methods of authenticating or importing numbers to what ever takes your fancy, our infrastructure won't cost you or your country a cent to make use of, and we have a proven system that just works and soon it will even be better!

Currently have the software running on a single server/IP for testing purposes, for those interested/wanting to play and try to break the code checking for security issues, please feel free, I think/hope I did things as securely/well as possible, but I tend to get out by one errors :)

# dig +short 5.5.3.8.5.5.5.0.0.8.1.e164.arpa any @208.70.151.162
200 10 "u" "E2U+SIP" "!^\\+1800(.*)$!sip:1800\\[email protected]!" .
200 10 "u" "E2U+SIP" "!^\\+1800(.*)$!sip:1641641800\\[email protected]!" .

Both UDP and TCP seem to be working correctly:

# dig +short 5.5.3.8.5.5.5.0.0.8.1.e164.arpa any @208.70.151.162 +tcp
200 10 "u" "E2U+SIP" "!^\\+1800(.*)$!sip:1800\\[email protected]!" .
200 10 "u" "E2U+SIP" "!^\\+1800(.*)$!sip:1641641800\\[email protected]!" .

Currently the code doesn't compress hostnames in queries/responses by using labels etc, and this does cause larger then normal DNS replies, and I'm really tempted to allow larger then 512 byte payloads as most DNS relays I've tested seem to handle this ok, although there is bound to be firewalls setup specifically to block this.

Or maybe people could add/remove their DNS server IPs from a whitelist of IPs that will accept larger then normal DNS packets, although DNSSec kinda does this already to get round needing to fall back to TCP every time a query occurs.
 
Holen Sie sich 3CX - völlig kostenlos!
Verbinden Sie Ihr Team und Ihre Kunden Telefonie Livechat Videokonferenzen

Gehostet oder selbst-verwaltet. Für bis zu 10 Nutzer dauerhaft kostenlos. Keine Kreditkartendetails erforderlich. Ohne Risiko testen.

3CX
Für diese E-Mail-Adresse besteht bereits ein 3CX-Konto. Sie werden zum Kundenportal weitergeleitet, wo Sie sich anmelden oder Ihr Passwort zurücksetzen können, falls Sie dieses vergessen haben.