Archives

DNS session
.
Wednesday 28 October 2020
.
11 a.m.
.
DAVE KNIGHT: Good morning. Hopefully you can now see the agenda screen. So, first up administration: Does anyone have any changes they would like to make to the published agenda?
.
I'm not seeing it. We have got previous minutes on here. We didn't have a meeting of the DNS Working Group at RIPE 80, so I just kept this in as a reminder to point that out so there are no previous minutes to approve. So, the last item of administrivia matters is our co‑chair selection. Joao has served one three‑year term, we are exercised our Chair election process, and we had one volunteer, which was Joao which has received consensus on the mailing list that he should continue for one more term. It was out that it would have been better if we got some fresh blood in, and so, so that point, I would point out that I term out in one year and we'll have an opportunity then definitely to bring fresh blood in, and at the next meeting we can make a bigger effort to point out that the benefits and fun times that could be had being a Chair of the RIPE database DNS Working Group in preparation for that.
.
So, if there is nothing else to administer, we can press straight on with our presentations. The first one today is from Anand from the RIPE NCC who is going to give us a report.

ANAND BUDDHDEV: Good morning, folks. I hope you can hear me now. Let me share my screen. I am afraid Google is asking me to quit before I can share my screen. Just give me one minute... hi again, I am sorry for that, I had to quit my browser and restart it in order to share my screen.
.
So, good morning everyone. This is Anand from the RIPE NCC, and I will be doing the DNS update for this session.
.
So first up, I'd like to start with an update on the secondary DNS service that we had with Neustar. So, we actually had secondary DNS service with VeriSign and VeriSign had sold this business to Neustar last year, and so our secondary DNS service was migrated from VeriSign to Neustar. We actually completed this transition in September last year and we were using Neustar's secondary DNS service, but we found that the query volumes at Neustar were going up and that's kind of normal, I mean, DNS query volumes do go up with time. But then in April this year, all of a sudden the volume increased substantially. The number of queries being served by Neustar were far higher than normal and there was a sudden jump that we couldn't explain and that Neustar couldn't explain. The effect of this was that we were going over our contracted volume and the usage charges were very high.
.
So, at this point, we began evaluating this service, and we felt that it had no visible benefit for us, and so we decided to withdraw secondary service from Neustar in July. So, all of the zones that the RIPE NCC manges, around 50 of them, we withdrew them from the Neustar DNS servers and in September, when the contract was coming up for renewal, we forewent the renewal. So, at the moment, the zone ripe.net and all its related zones, all the zones that we manage for providing RIPE NCC Services, these are secondaried by the other RIRs, AFRINIC, APNIC, ARIN and LACNIC.
.
Next up I'd like to talk about k.root, so, since RIPE 17 last year. We have added 11 new instances and, as of this meeting, RIPE 81, there are 81 active instances of k.root. So that's just a happy coincidence there, 81 and 81.
.
The k.root cluster all together has an average query rate of about 120,000 queries per second and, during busy periods, this can peak at 200,000 queries per second. This is far lower than the actual capacity of the cluster, so, the entire k.root cluster is quite well provisioned at the moment.
.
Next up, I would like to talk about our authoritative DNS service. This is the second DNS cluster that we operate which carries the ripe.net zone. It also carries all the other zones that are managed by the RIPE NCC. It's also the cluster where we publish delegations of all the address space that the RIPE NCC manges.
.
So this is an independent cluster with its own AS and its own servers and everything. And this year we have been working on expanding this service, and this is even more important now, because we have cancelled the secondary DNS service with Neustar, and it's imperative that our own DNS cluster has the capacity to answer all the queries coming our way.
.
So, we have bought new servers and routers for the core sites and we are currently busy with the upgrades. We are going to be adding more capacity, newer servers, we are also replacing some of our old Cisco routers with new routers from Arista, and one of the others things we have been doing with this cluster is that we also want to add more hosted instances. This is where a host in our community offers us a server, and similar to k.root, and we then deploy an authoritative DNS instance on it and it becomes a full Anycast clone of our authoritative DNS service.
.
So, this year, NaMeX offered to host one such authoritative DNS server in Rome, and a Norwegian company called JCloud offered to host such an instance in Oslo, so we have a total of three hosted instances, the third one being in Vienna hosted by Anexia IT.

I would like to then talk about name server diversity. We have mentioned before that at RIPE NCC we run not just one name server but we run BIND, not DNS and NSD on all our clusters, and we continue to value this diversity. And a recent example of this was that there were bugs in two of these implementations, and they caused the name server to crash. And this can happen to any software of course, there are bugs in software, but fortunately, when one of the implementations crashed due to an update in a zone, the other two implementations continued to function and our clusters were not affected, service continued as normal without any down time anywhere.
.
So, we value this diversity because these kind of bugs can affect any software, but they become less critical when you are running a mix of servers.
.
And this actually justifies the effort of understanding packaging, configuring and maintaining different implementations, it causes us more work because we have to have different configuration templates, we have to package all of these, we have to keep up to date with new versions, but this effort is justifiable for this reason.
.
So, I'd like to talk about DNSSEC. All the zones that the RIPE NCC is maintaining are currently signed, and we are using algorithm 8 for this, RSASHA256. However, RFC 8624 was published, and this RFC recommends upgrading to an elliptic curve algorithm, Algorithm 13. There is no immediate need for any upgrade, but the RFC suggests that all those that were using algorithm 8 should be looking at upgrading to Algorithm 13 or 14.
.
There are some ccTLDs and gTLDs that are now signed with Algorithm 13. They have ‑‑ the operators of these TLDs have talked about their experiences. There is nothing difficult about this. The algorithm rollover is quite straightforward, and there is wide support for verification of Algorithm 13 signed zones. So we believe the time is now right for the RIPE NCC to also rollover to Algorithm 13.
.
We are developing a plan this year, and our intention is that somewhere in the second quarter of next year in 2021, we plan to upgrade to Algorithm 13 for all the zones we maintain.
.
One of the other things that we had started working on last year was on support for RFC 8078, this is also known as the CDS /CDNSKEY. This key publishes an mechanism to that allows for automatic updates of a child‑signed zones record in the parent zone and this is good because it eases DNSSEC deployment. Unfortunately, the work on this has been delayed because of other priorities this year, and so we are aiming to have this implemented by mid‑January 2021.
.
Finally, I would like to make a mention of DNS Flag Day 2020. This is an effort by operators and vendors of DNS software to change the defaults in DNS software to better work around the problem of packet fragmentation. So, the recommendations from DNS Flag Day 2020 for authoritative DNS servers are to ensure that they are able to answer queries over TCP. And all of the RIPE NCC's DNS servers already do this, they answer queries over TCP.
.
The other recommendation is to set the EDNS buffer size to 1232 on authoritative name servers. And this is to ensure that the name server does not emit large packets, packets that would be fragmented and potentially not reach the client.
.
So Knot DNS has been doing this for a while. The EDNS buffer size in Knot DNS has been 1232 for quite some time, and the latest releases of BIND 9.16.8 and of NSD 4.3.3, they also default to an EDNS buffer size of 1232.
.
It is our intention to keep these defaults and so as we upgrade BIND and NSD on k.root and our authoritative DNS clusters to these latest versions, they will be using this lower EDNS buffer size. We do not anticipate any problems here because we have not observed any of our DNS clusters sending out responses larger than approximately 800 bytes. So, we actually don't expect any major problems as a result of this change and we expect things to continue working just as they are.
.
And with that, I come to the end of my presentation. Thank you for listening. And if you have any questions, please ask away.

SHANE KERR: There is three questions that people have asked in the Meetecho. We have only got two minutes left though, so please be brief. The first question is: You mention that the query volume rose substantially in April. Have you looked into why this happened? Is it related to lockdowns?

ANAND BUDDHDEV: We did try and look into this. We also asked for help from Neustar and their technical team also looked into this. We couldn't quite figure out why the query rate suddenly increased. The focuses were somewhere in Asia mostly, but there was no clear correlation with Covid. It looks like it should be, but ‑‑ and the timing seems a little bit too coincidental, but we don't know. We don't know why the query volume went up.
.
The other interesting thing was that it went up at Neustar servers but not at the other name servers. The cluster that we operate, the query rate there, you know, did not go up substantially.

SHANE KERR: Although I guess if it's in, if it was in Asia, then it might be something to do with how the Anycast is set up or something like that.

ANAND BUDDHDEV: Sure, sure, we can't rule that out. But unfortunately we don't have a clear explanation for this.

SHANE KERR: Okay. I have a related question, I am going to sneak my own question here. Did you see, once you stopped using Neustar did you see a corresponding increase in queries to your other servers?

ANAND BUDDHDEV: Yes, there was an increase in the query rate at our cluster. We saw that in our graphs. But it was not substantial. In fact, you know, the query rate went up from something like 800 queries per second for these zones to something like 1,000 queries per second. So, there was an increase of something like 200 queries per second, which, for our cluster, is nothing.

SHANE KERR: Okay. Great. Well, I think we're going to have to leave it there. There were a couple of more questions, people can e‑mail you or chat in the chat room. Thank you.
.
So, I guess I'll go ahead and introduce our next speaker since I have got the video and mic here. It's going to be Geoff Huston and he is going to be talking about measuring query name minimisation.

GEOFF HUSTON: Thanks, Shane, and coming to you from this black void tonight. What fun!
.
I'd like to talk about measuring query name minimisation, and I'll not actually talking about query name minimisation and whether you should or shouldn't do it but what we're trying to actually do it understand how many folk are doing it. For those of you who have been sleeping under a DNS rock over the last few years, what query name minimisation does in about ten seconds is instead of asking the full query name as it tries to find the right authoritative server, a recursive resolver will trim the query name down to match just the name of the server, sorry the same of the zone, the server is authoritative for, plus one extra label. And it will use an NS query type.
.
(No slides)
.
So the original diagram, use the full name, query downwards, now, it's an NS query and it's trimmed. So, you know, go look, go read, whatever, that's the process. Don't forget, this is recursive to authoritative, it doesn't affect stubs, it doesn't affect other parts of the DNS. This is really just trying to address the information leaks  ‑‑
.
(Slides from here)
.
I said that! Who implements query name minimisation? Well of the major resolver implementations of recursive resolver, all of them ‑ BIND, Unbound, knot and Power DNS ‑ actually implement it. So, as long as you are actually up to date, that's a big thing, or, you know, you are running a standard version of this software, then you should be doing query name minimisation in your queries. So what we're doing here is actually doing a measurement using the end user, so what we do is we cede a domain name in the end user that actually has a number of elements and we are actually looking to see if the query to resolve that compound name is actually resolved with query name minimisation or not.
.
When we first looked at this a year ago, the results were kind of disappointing. We did half a billion tests, which we thought was a pretty big number, you know, and, of that, we found only 11 million, or 3%, 3% were actually doing this. Very few of the queries were NS, so, it seems that people thought the RFC was kind of wrong and, oddly enough, NS was giving some weird answers, so folk were migrating to using an A query and there were a few hangouts using AAAA. So 75% of the queries for the A record, 25% for the NS, and that was 2019, the year is not 2019, this is the horror year of 2020, and, in the horror year of 2020, we got these results.
.
Slightly fewer measurements, a mere 350 million. What we found now, though, is the query name minimisation rate for users as a percentage of users has gone up to 18%, one in five or thereabouts, which is actually really good.
.
The amount of folk doing NS queries has dropped off the deep end. No one does AAAAs because v6, and everyone, 94% of these query name minimisation queries use the A query type, which seems to be in vogue.
.
It certainly points to a pretty rapid uptake in the space of twelve months. You'd think because it took a while to measure it, that in the period of two months it would rise, but you'd be wrong. Things were looking slightly better in August than they were certainly looking a couple of days ago in the third week of October. Oddly enough, the rates are actually falling, not growing. How bizarre! I don't know why, I have no explanation. But maybe there is a few more clues when we look at the profile of who is actually doing it.
.
This is ranked by the percentage of users who are geolocated into that country, who we observed doing QNAME minimisation. And there aren't many. I hesitate to say any of the top‑end countries that include both Greenland and the Democratic People's Republic of Korea, North Korea. I am like, this is ‑‑ it's such a curious set.
.
India. Where is European mainline countries? Well, Germany is a little bit lower than, France, etc. Where is the US? Well, not there. Where is Australia? There, there is a bit of New Zealand way down. What an odd set. It's almost as if, in some ways, these are the folk that are actually writing the latest releases of DNS resolver software. Interesting.
.
Let's talk about resolvers, because I just mentioned the word. When we talk about the DNS, it's really, really hard to talk about resolvers because quite frankly we don't know what they are. The old model that a resolver was this sort of engine in a piece of paragraph running a piece of software is just completely overtaken by events and almost all the resolvers that we see sit inside a common sub‑net with a bunch of others leading us to believe that the model these days is some kind of front end doing some kind of query splaying using rules of its own invention because this isn't standardised. An engine gets picked and then it does the query who is doing the query name minimisation? Difficult question. In theory, one of the engine gets a full query, it should resolve it itself, but some of these behave pretty oddly. We don't know what a resolver is. When we say we are talking resolvers, we really don't know what we're saying.
.
But what we did do is bucket these up by the open resolvers that we understand who they are. Google doesn't do it, not in the way we were doing the measurement. In fact, the only folk that did quite clearly is CloudFlare at 85%, and even open DNS at 71%, seems to be pretty much there. But how do you get a 50% number?
.
These are big Anycast networks. But why are only half of the people using CloudFlare? Actually showing QNAME minimisation? Is this this Anycast Cloud isn't what you think it is and some bits of the Cloud do and some don't. Some engines do and some don't. I have no idea. And quite frankly, only looking at the authoritative servers as we data networks I can't tell you the answer. But when I see these 55, 56% sort of answers, even 67% from quad 9, I kind of wonder, because it's really hard to understand why an open resolver where you think you get consistent answers doesn't give it to you.
.
These are the largest ISPs and whether they do query name minimisation or not. Obviously China is a very big country and the ISPs are enormous. India has a lot of people, a large provider. Half of the recursive resolvers in reliance geo, 56% do and the other half don't. Similarly, China Mobile in the Jiangsu province and so on. A lot of these answers raise more questions than that about how uniform these resolvers are in this kind of behaviour.

So some observations. Yes, it's growing, it's really heartening and while all the common vendor code has actually implemented it we don't see it as much as we think we should. Because if everyone's has implemented it and you actually need to turn it off, then why is it only at 18%? Cheats the concern or the margin going on? Why is it not everywhere right now?
.
Now, there is one part of this that becomes obvious when we think about Google, because Google do do query name minimisation of a sort. Because they only do it for the first three labels and guess what, we are using unique labels in the fourth and fifth level and Google says, no, you are too low in the tree, I don't actually give a stuff about your privacy.
.
Why, oh why isn't privacy important all the way down? Am I a week sort of scum who inhabit the lower orders of the DNS unworthy of such consideration from recursive resolvers? Is it the load in uncovering the name servers different with query name minimisation? I don't think so. Or, is Google really only with holding information from the TLD servers in the route as an act of spite in some ways that this is not looking after my interest (root) it's simply with holding information from others. That's highly likely. Or a recursive resolvers operator is just making it up, yeah, we thought of the three rule, it was a Wednesday ‑ three? I have no idea.
.
More questions: Who should be doing? When is it important? Is it important that Google do it or not? Because by the time a quad query gets to Google, you are hiding in the crowd. One fifth, the entire Internet user population sends their DNS queries to Google, so when Google emits a query, who made that query? One of billions, take your pick.
.
So, in some ways, maybe it's more important for small than big. Maybe. What about ISP operated recursive resolvers? Where is it important for? And, quite frankly, is the query name itself a problem or is it the combination of, I know it's that recursive resolver that did it and this query? Is that the problem? And as soon as you add client sub‑net, is this entire thing moot because this that has blown all considerations of privacy and there is no coming back.

So the last question here is it's a question to you, dear audience: What's important in privacy? Now, I think ‑‑ well rank your own, quite frankly for me, client sub‑net is a disaster on legs, it should never have been implemented, ever, ever, ever. It just leaks your information and mine all over the net like a sieve and query name minimisation can't stop this horrendous leak, so you know that's top of my list.
.
What about the other four? Which of these actually offer bang for the buck? Which is worth concentrating on? Which is so marginal? I don't know. Give it your own answer.
.
Thank you. I'll hand it back for questions.

DAVE KNIGHT: Thanks, Geoff. I'll see if we have questions.

GEOFF HUSTON: From Chris: Which vantage points were used for the expertiments?
.
We used an online ad. We enrol users from everywhere as long as they see ads. If you have seen YouTube lately, you have seen this, anyone who gets an ad is probably going to get measured, including the good folk in the DPR Korea. Where are we aiming? Don't forget this the recursive to authoritative and we run three authoritative systems, one in each major area of the globe. The latencies aren't that bad, they are not brilliant, but in some ways because we are testing people and the resolvers they use, it actually doesn't matter the vantage points on the authoritative side, so, yes, the other side, the user side we're measuring, well, if you have seen an APNIC ad, A) don't click it because we pay more if you do, and B) you are running a measurement for us and we thank you.

DAVE KNIGHT: We have Lawrence Lehmann is in the audio queue. Open your mic.

AUDIENCE SPEAKER: Hoping that ‑‑ yes, you can hear me. Hello, Geoff, this is Lawrence Lehmann. Have you looked at all into technical consequences of doing query minimisation? Do things happen to the DNS system in general in the large scale? You know, do query patterns shift ‑‑ what can we expect from this?

GEOFF HUSTON: So, the only place where there was some degree of controversy was actually in the behaviour of some content data network providers who were doing quite complex C name mapping when they got a target name and they were moving it to an alias where the domain name was sitting inside the CDN itself, and that C name mapping really had some issues with partial names, and if I was going to go a little bit further with my snide comments about Google and trimming, my suspicion is that we only trim down to three labels, actually avoids the problem of the C name mapping inside CDNs.
.
The other issue flying around is actually emptying on terminals and what happens when you ask an abbreviated query where the string inside the zone is actually A dot B dot C dot D, it's a non‑delegated stream. If you only ask for the A part it's an empty non‑terminal. It's not an NX domain. The answer should really be keep going, expand the name form. How many folk do compound names inside DNS zones? I don't know. Is that a problem in QNAME minimisation? It shouldn't be, as long as your authoritative name servers actually understand what's going on and give the correct answer to a partial query.
.
So those were the things and they were largely sorted out over a year ago, in fact two years ago, when we shifted from NS queries to A queries. Because that actually gave a behaviour that you know, resolvers that would form in in QNAME minimisation could understand and press on. I hope that answers you.

AUDIENCE SPEAKER: Yes, but it just struck me, isn't that thing with the hidden alias, isn't that the reason for ‑‑ or, sorry, the hidden non‑terminals, isn't that the reason for not using query minimisation with more than three labels?

GEOFF HUSTON: That's not a good reason for not doing it because, in essence, most recursive resolvers and most authoritative servers, the authoritative servers give the right answer, press on, keep going, don't stop now, which is really what you are trying for. The issue is: Is that less efficient in those situations? Yes, it is less efficient, because the thing about the full query name was it was a massive privacy leak, but you weren't trying to the solve find where the zone cut happened, because if there wasn't a zone cut it just worked. So, that's the trade‑off here.
.
On the other hand, don't forget that a large amount of folk, not route server operators, but a whole bunch of other folk who are privy to your query, make a business out of this. Will this stop it? Well, yes and no. It will stop authoritatives who are high in the tree knowing what's going on lower in the tree. Recursive resolvers always have all the information and QNAME minimisation doesn't alter that picture of information. The recursives are all productive to all your secrets.

We should be running out of time by now. I will turn off my slide here. Thanks, all.

DAVE KNIGHT: Last up today we have Andrew Campling talking about development in encrypted DNS.

ANDREW CAMPLING: Can you hear me okay?

DAVE KNIGHT: We don't see anything yet. There you go.

ANDREW CAMPLING: Welcome, everyone. I am just going to give you an overview of some of the more recent developments in encrypted DNS, so following on nicely from Geoff's always fascinating presentation and his last point about what aspects of privacy are important to you.
.
So, just by way of very brief background on this matter. Just as a reminder, there's been pressure to encrypt DNS as part of a drive to have full end‑to‑end encryption, partially because of allegations of abuse of DNS data in some quarters. I have to say, in Europe, that appears not to be backed up by hard evidence, that I'm aware of, because of GDPR, amongst other things.
.
Of course, DNS over TLS or dot has been around for a while now, but doesn't seem to have had enormous adoption, although clearly there are pockets of it, and there has been a drive to enable applications to have direct access to DNS. And knitting all of those together, that saw the evolution of DNS over HTTPS or DoH with a standard ratified by the IETF in late 2018, RFC 484, for those of you interested in that. Remembering it is just a protocol. As defined, it doesn't include anything around resolver discovery or resolver selection. It does indeed protect DNS queries from being monitored by third parties inserting themselves in the connection.
.
But, because there are always trade‑offs, it can have an impact on a legal content blocking, malicious content filtering, parental controls, DNS, etc. Which depending on your point of view is a bad or a good thing. So I make no comment on that, I just reference that there are some issues there.
.
In terms of where we are in the availability of client software actually using encrypted DNS. Some of the more recent announcements, Firefox was probably the first high profile adopter of DoH in the browser, which is implemented by default in the US, and I put on the screen the dialogue box that pops up if you are using the US software building which encourages you to just click the blue box to say, okay, I have got it and move on. The first implementation simply supported CloudFlare. More recently, next, DNS was added as a pre‑configured option and, in the last few months, Comcast was added to the list as well. There are currently three by default, although you can of course configure manually should you choose to, although there is evidence of very few people actually do that.
.
Moving on through this year, Chrome added support for DoH in mid‑May of this year. It uses an auto upgrade facility as the primary mechanism. So if it detects that the DNS operator that you are using is on its auto upgrade programme, then it will automatically switch you from your regular DNS over to DoH in the background.

There are some issues with that in that the process being used doesn't work well with many of the resolvers used by European ISPs, that's because of the architecture that is sort of commonplace in Europe, which is DNS forwards with a private IP addresses, and that's an issue which impacts elsewhere, which I'll mention as well in a moment.
.
And it was interesting in the discussions in the IETF that clearly a lot of software developers were unaware of that architecture having much more familiarity with practices in the US rather than other locations.
.
Bringing us more up to date. Apple announced support for DOT and DoH at WW DC 2020 mid this year, and it was switched on in iPad OS 14 in the last few weeks and it will come on in Mac OS Big Sur, although I don't think that's shipped yet. Because it's in the operating system level, there are configuration options for enterprises, individuals and applications to use or not use encrypted DNS.
.
Finally, on the client side with Windows 10, there is support for DoH in Windows inside a programme, again with an auto upgrade facility much like that used in Chrome where it detects the DNS and has a lookup table, and again it suffers from the same limitation that that doesn't work well with the architecture that's come in place for European ISPs.
.
I have heard a few suggestions that that might actually ship. There is a full release for Windows 10 in the first half of 2021, but that's not a date that's been confirmed by Microsoft thus far.
.
Where we have with the IETF and the standards development. The DoH Working Group finished last year. A new Working Group, the Adaptive DNS Discovery Working Group, or ADD Working Group, was formed in February of this year to focus on resolver discovery and selection, to build on the protocol itself and based on the charter, and I put the quote from the charter, is, it is intended to deliver sort of discovery and selection methods that work across a variety of different network environments, which I have listed on the screen.
.
Most recently, discussions in the Working Group have focused around agreeing the use cases and the associated user requirements to go with those, and it's again true that not all the proposed discovery methods would sort of work well with the architecture currently used by many of the European ISPs.
.
But it's early days for the Working Group and there is going to be a lot more discussion next month at IETF 109, and then I think a lot of the actual work with start to bed down. So there will be more on that over the coming months.
.
What the Working Group can't cover, because it's explicitly ruled out in the charter for the Working Group, is to look at any policy matters related to DoH. So, that discussion needs to happen in other places. I should mention also that a recent publication this month from the IETF RFC 8932, helpfully has published recommendations for DNS privacy service operators, which is also relevant to those resolver operators that are implementing encrypted DNS. So that's definitely worth a look if you have not seen that.
.
But in terms of where the discussion can happen on policy matters if not in the IETF Working Group, there has been some discussion in the IGF and its satellite fora. So if you participated in EuroDIG mid this year, one of the workstreams did sort of discuss on the policy matters linked to encrypted DNS, and if that's of interest to you, that's all recorded, so it's worth going to the EuroDIG site, you can access that.
.
And at IGF itself, IGF 2020 coming up next month, again there is a workshop focused on DoH, so that's worth registering for if that's of interest to you.
.
And there is also the encrypted DNS deployment initiative, or EDDIE, where many of the participants in fact who are involved in the ADD Working Group are also registered for EDDIE, and that's a group set up by some of the guys at Comcast initially to provide a forum for further discussion. So, that is well worth getting involved with if encrypted DNS is a matter of interest to you.
.
Just briefly touching on some of the other developments. On the ISP side, I mentioned that Comcast is part of the Firefox programme now, and is also on the auto upgrade programme with Chrome. So Firefox has probably been the most proactive of the large ISPs to date. And it's starting to see some reasonable encrypted traffic, it's not a huge percentage, I think it's less than 1% of their billion DNS queries a day, but it's growing.
.
Deutsche Telekom here in Europe has got a trial implementation, as has BT group. Both of them, it's early days with their implementations. They are not seeing much traffic and that's primarily because they are not getting traffic from the sort of main clients out there. You can configure the clients manually, but very few people do that. So, because it's not done automatically, they are seeing very little traffic currently.
.
In terms of resolvers policy. Mozilla has probably got the best known activity in this area with its trusted recursive resolvers, or TRR programme. And they were certainly very early to get out there with their programme, and I have recently been working with various people in the industry to develop a European DNS resolver policy document which will be available shortly to have a policy developed specifically for Europe in the context of GDPR. If anyone is interested in that, by all means contact me directly.
.
And I should also just remind you that ‑‑ sort have sort of ceased in this area, so encrypted client hello (ECH) is under development in the IETF's TLS Working Group and that will give you encryption of the SNI data to continue the journey of removing the amount of data that's sent in the clear as part of the overall sort of programme within the IETF.
.
I have put some links on the slide. One for the ADD Working Group, if you go on that link, that will sort of take you to sort of details of the sort of meetings, give you access to various materials and so on, and it's well worth doing that. There were eight papers presented at the IETF 108 to the ADD Working Group in July, two interim sessions were held in September and we got two sessions of the Working Group scheduled for IETF 109 next month, so there is a lot of activity there. And I put a link also for the Working Group mailing list, which is well worth joining if you haven't already done so.
.
I mentioned EDDIE. That's free to join. I have put a link for that and its mailing list on the slide, and also to the sort of GitHub that EDDIE uses for its work streams. And then finally, I hold a weekly call for matters related to encrypted DNS, which again is free to join. If you want to join that, just e‑mail me and I'll add you to the mailing list and send you an invite for that weekly call.
.
And then last point, a quote from Jason Livingood at Comcast. Basically, I won't read it out, in full, but essentially he is encouraging resolver operators to consider adding support for encrypted DNS now while volumes are low, so it's a relatively low risk thing to do, so you can actually sort of test and deal with all the wrinkles before traffic restarts to ramp up, and I have heard that view expressed by other resolver operators, it's just he happened to put it on a mailing list with a very handy quote, so I took the liberty to share that with you.
.
So if you are a resolver operator, if you are not yet into encrypted DNS, have a look now and just start to get familiarity and look at enabling deployment.
.
And that's me. I think I have finished within time, so if there are any questions...

DAVE KNIGHT: I don't think we have anyone in the audio queue, but we have a couple of questions in the Q&A.

First one from Ali: Is there any report or study about DNS encryption load on CPUs?

ANDREW CAMPLING: There have been conflicting reports on that, from the most recent discussions, certainly that wasn't a concern that Comcast had in terms of it being a significant load. I think some of the early modelling suggested it was questioning how scaleable it would be. I know BT did some work on that, as did Akamai, but I think the most recent sort of reports from those actually deploying in the fields seemed to be suggesting it's not as bad as the modelling had originally suggested. That's the most recent material that I have seen on that over the last month or so.

DAVE KNIGHT: (On mute) this is from Thomas Schafer: How about conflicts, DoH, DNS 64, are there any studies about it? I know there are some providers but it has to be set manually.

ANDREW CAMPLING: I don't know the answer to that, I am afraid. Not that I am aware of, but not that I have come across.

DAVE KNIGHT: Okay, this last question is from Benno Overeinder, not a question but a comment: The DNS Open Source software developers, have those support in their most recent resolver or proxy resolver, ISPs have different options to implement DoH support in their own network?

ANDREW CAMPLING: Yes. I think that's clearly true and certainly I know a number that are using Power DNS in particular, I think Quad9, I am not sure about Comcast, I think DT and BT both are using Power DNS. So, yeah, and clearly other vendors are available. I should also have mentioned by the way that tools are starting to be developed now to detect (to detect the DoH streams, which is inevitable so I'm hearing that there are some tools out there claiming a 95% rate in at the equity itting the presentation of DoH streams which I'm sure will find their way into enterprise tools where enterprises wish to block the use of DoH but to ensure that people use their proxy servers for privacy policies, etc. So I have a feeling that will be developed and you are going to see some more of in the next few months.

DAVE KNIGHT: That's interesting. So your slide is quite dense with information. I'll point out to everyone if you want to grab the slides, they have been all been uploaded, you can find all of those slides. Thanks to all of our speakers today, and I get a thank you to Joao for offering to volunteer to continue in the Working Group Chair for another three years, and I think we have one minute left if anyone has any other business to bring up.
.
I am not seeing anyone in the audio queue or anyone in Q&A. Okay, I don't see any other business, so, with 30 seconds to spare, we can wrap this up. Thank you, everyone, for attending and again to all of the speakers ‑‑ oh, we intend to continue our remote DNS Working Group sessions, still doing those about every three weeks, I think the next one will be on November 18, so I hope you can make it along to that. We will send information in advance to the mailing list. Thanks again until next time.
.
( Lunch break)