The following are the outputs of the real-time captioning taken during the Fourteenth Annual Meeting of the Internet Governance Forum (IGF) in Berlin, Germany, from 25 to 29 November 2019. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record.
***
>> MODERATOR: Thank you, everyone for joining us in the room. Thank you to those online who are joining us.
Thank you, all. Welcome to our workshop to address what is a very topical issue, and it's to do with law enforcement access to user data in the context of criminal instigation. My name is Alexandre Roure, I'm from the Computer & Communication Industry Associations. CCII is co-host of this panel and our second co-host is the Council of Europe and Alexander Seger. We are delighted to have Alexander Seger with us as co-host, but also as speaker and present some of the views -- some of the developments happening in the Council of Europe.
I'll be your moderator for the purpose of this workshop. Rachael Stelly, my colleague from our D.C. office, will be the online moderator and will take questions from the online audience when we move on to the Q&A session.
CCIA and the Council of Europe, that's not the first time that we are co-hosting a panel on this very issue. We did one in Mexico. We did another one in Geneva two years ago. There was a bit of a gap last year in Paris, but now we're back and we think it's time to catch up.
Since that time, a lot has happened. We have seen the U.S. moving forward on legislation to reinform law enforcement access to data. The European Union is also doing the same. The Council of Europe now has been discussing for about two years now, a framework to deal specifically with that issue and adding a second protocol to the Cybercrime Convention.
I think we've touched on many of these issues. There are many developments out there. I want to first introduce the panel. And maybe set the scene very briefly, just to let you know. We'll have first five minutes for each speaker to present their brief introductory remarks.
After that, we'll move onto a discussion among panelists and then will open the discussion to all of you in the room and to those online as well.
So first of all, as a first speaker, we have Fernanda Domingos, she is the federal prosecutor in the cybercrime units of the Brazilian Prosecution Service. So she'll offer her perspective from the law enforcement community on how data has become so important and how Brazil, among other countries, perhaps is reforming its rules to empower the law enforcement authority to get easier access data and to conduct its job thoroughly.
Then we'll hear the perspective of the business community. Ludmila Georgieva is public policy manager at Google in the Brussels office. Her job is to do with -- she basically -- she's very much hands-on. European discussions around law enforcement access to data, as well as international developments as well.
Next up will be Jennifer Daskal. And Jennifer Daskal is a professor -- I'm sure most of you know her. She's a professor at the American University, Washington. And she is leading the program on tech law and security. Is that correct? Is that the name of the program?
OK. Sorry. Thank you. We'll hear as well Bertrand de La Chapelle, who is the cofounder and executive director of the Internet and Jurisdiction Policy Network.
The Network has been doing a lot of work for some time in trying to engage with a multitude of stakeholders to try and design a common -- well, to come to a common understanding of the problems. To companies, sort of like the legal and policy discussions in other fora as well, precisely on that issue.
And finally, we'll hear from Alexander Seger with the Council of Europe. Alexander is the head of the cybercrime division in the Council of Europe, and he leads the discussion on the drafting of second protocol to the Budapest Convention, to the Cybercrime Convention.
Just two words before we move onto the discussion, to the presentation of the speakers. Clearly, we've seen over the past few years we have seen countries that are -- access data in criminal investigations.
What we are witnessing are oftentimes unilateral assertions through national legislation.
But these have oftentimes implications for foreign countries, for the residents of those countries. And that includes, among other things, their privacy rights and the protection of rights by third country legislation. We'll look into that with all of our speakers.
One thing I should add that naturally, there are also implications for the business community, for those online service providers that are used by everyone in this room as well as potential criminal suspects, but also innocent people.
And the business community is really very much the custodian of user data. So we'll explore with the panel the room for maneuver. The extent of company's responsibility and liability when they receive law enforcement data access, as we see more and more reforms across the world to better equip the law enforcement community.
I think I'll stop here and Fernanda, if you could please, just five minutes of introductory remarks. That would be fantastic.
>> FERNANDA TEIXEIRA SOUZA DOMINGOS: Thank you very much. Very honored to be here at this distinguished table. Well, from a law enforcement perspective, nowadays, I can say that almost every crime needs digital evidence. Not only crimes committed via Internet or via computer systems. If we come across a home site, besides the crime scene, we go for the cell phone, for the computer data, for the victim services on the cloud. Law enforcement, primarily, needs information about IP addresses, which allow knowing the origin of a connection to the Internet.
And it may point out to the author of a crime, after the unfolding of more investigation. So the majority of investigations start with the IP addresses. The law enforcement, also we use traffic data and content as evidence to go on further investigation and to support persecution in court. In Brazil, we already have good legislation in force. It's our Brazil civil rights framework. Which was inspired in the Budapest Convention. That gives law enforcement enough power to get data from the ISPs whose services are targeting our territory. As long as we have a domestic judicial order. By the way, it is in concert with the EU evidence proposal.
Considering safeguards. A judicial order in Brazil is required to get access to can any data, IP, traffic, or content. When it becomes a criminal action, the defendant has access to all the evidence gathered as well as to the chain of custody. Being able to challenge the evidence in court and even plea for repair in a judicial redress action if he believes there has been any kind of abuse.
Nonetheless, we still need to access data that is outside our jurisdiction, since more and more criminals are using the facilities of foreign Internet services.
The MLA framework has to be improved because the way it is now, it is not at all suited for digital evidence that can be moved and disappear in an instant.
I'd like to illustrate that, because we had child abuse case on an Internet forum, where who is pointed to go -- when it didn't have any office in Brazil or targeting issue. We issued MLA to the U.S., and it took a year. And we have to issue another MLA to friends where the crime was and another one to the Netherlands where the Internet host was.
That means that the child was to continue to be abused for four more years before we would get any hint of the information and little probability of the evidence to be still there.
We still have problems with ISPs that refuse to comply with domestic judicial orders concerning, for instance, hate speech. In Brazil, we had the first case of terrorism, which according to our law, includes preparatory acts. We were lucky, because due to other means, we got to know what the terrorist group was meaning to do.
But two years before, we already was investigating a Facebook profile where one of these men that were arrested was spreading hate speech and already recruiting for terrorism purposes. But we couldn't go on with the investigation, because information about this profile was refused to us and was not to come even via MLA, because it offends the U.S. free speech amendment. That means that we wouldn't be able to give law enforcement response according to the laws voted by a free elected Congress, putting at risk people in my country, because of a foreign legislation not accountable to the people of my country.
Anyway, just to finish. In Brazil, we are very attentive to the discussions on EU evidence proposal and also eager to know about the second additional protocol to the Budapest Convention developments and the way this discussions will be able to help law enforcement to get access to data.
We are still skeptical about the cloud act solution. It does not solve the hate speech problems or terrorism incitement problem. Considering U.S. law would only allow the disclosure of data in these cases if they met the Brandenburg case standards about imminence. What in general, is too late for us.
So I would finish saying that from a law enforcement perspective, it's rather frustrating not being able to act to stop ongoing crimes. Not to arrest criminal due to this mismatch among the actors related to cross-border digital evidence.
>> MODERATOR: Thank you, Fernanda. I will turn my attention to you. Google is a truly global company operating in multiple jurisdictions and a slew of -- based on your transparency report. Can you tell us a bit of implications of what that means for you as a business and for your customers, your users?
>> LUDMILA GEORGIEVA: Thank you, Alexander and thank you for inviting us to be here. It's always a bit of a tough exercise to speak after law enforcement. So we'll try to be a bit solutions oriented. I think for us as Google, what has always been important was that on the one hand, of course, the law enforcement cooperation, to work with law enforcement. And on the other hand, and this is extremely important for us, fundamental rights in place, which are the balance between it.
So for the -- we receive, as you said, a lot of requests, 100,000 requests worldwide. And that's why we are also in favor. We have internal policies. As you mentioned the transparency reports and so on. We're also trying to develop best practices for industry. How to approach all the different questions.
But of course, kind of our framework, international framework, is always helpful. And one of the reasons why we -- the Cloud Act, we welcome the Cloud Act, we also welcome the European efforts for the eEvidence proposal, because it's harmonized. And the question is how you harmonize it. I think there is a balance between the different interests. We all want to prevent terrorism, and pedophile, but on the other hand they can be used for other purposes. Because once you disclose the data, the intrusion in the data subject rights has been already taken.
So the point is where we always have to balance it. That's why I think there is a lot of elements which, for us, we see important. One of the questions of jurisdiction, it will be interesting to hear about the Council of Europe and the Budapest Convention. I think the question of notice of the user. There's something under the U.S. Cloud Act that actually it is the authorities or the providers are obliged to notify the user, unless there are other circumstances which the law enforcement have to explain.
We have developed quite efficient tools of this user authentication, and in the U.S., there was also huge debate on the question of should we use this or not. We think this is a very important safeguard for user fundamental rights, but also for law enforcement for the future when there is any kind of a case afterwards.
I think the question of sovereignty, especially in the European Union and the question of national security is always raised when we are approached by one authority to disclose data of citizens of another member state. And I think this is the question.
When exactly this kind of a use notification could also serve to safeguard the rights of users and of national authorities. The question of -- I think the principle of necessity and proportionality -- also, of course these extreme cases of terrorism and pedophile cases.
But I think this is a border where I think all society will agree that we all should work together. But these are not the only cases we're facing. That's why I think the question of proportionality and accessibility are very important, but also the timelines to respond to certain requests are important. We can, as a big company can have a team in place that(?) everything. But for smaller providers, the question how to respond and to deal with requests. I think the question of harmonization is very important, but also the question of points of contact. I think this is a system that has worked very well when law enforcement authorities have the point of contact, and this person is trained to deal with all these requests, trained to work with different providers.
And this is something also, a very efficient model that we have experienced in different countries. And of course the question of material scope, which has been raised. The differences between the countries.
We're in the middle of everything. So we would like to have -- so very important issue is also how to prevent the conflict of law between the different jurisdictions. So this is also element that we are dealing with.
And any kind of a legal solution, we're very grateful for that. So I'll just make here point and looking forward to the discussion.
>> MODERATOR: Great. Thank you very much, Ludmila. Jennifer, it seems over the last few years the concept of jurisdiction -- I mean clearly, there's an overall recognition that perhaps the old system of legal assistance may not be hit for purpose. That's what we heard from Fernanda and we see more and more countries passing legislation. We see discussions in international fora. And clearly it feels like the concept of jurisdiction and territoriality are shifting.
Can you tell us a little bit about that and your views on this topic?
>> JENNIFER DASKAL: Sure. Thank you. And thank you for inviting me to be here and for all of you for coming and showing up. So I think just to step back a little bit and put what we've been talking about in context. Obviously, we all know that with the growth of the Internet, data moves across borders with rapidity, it's often mirrored and held in different places. Data transit, it's often broken into small packets and the location of data may be completely disconnected from the sovereign who is seeking -- has an interest in that data for a host of reasons, in ways that are very different than used to be the case with criminal investigations.
So when we think about criminal investigations in an analogue world, mostly law enforcement investigating local crime and local witnesses and local perpetrators and local victims, is dealing with evidence that is held locally.
Increasingly, evidence needed sought in the investigation of specific crimes including local crimes, involves some form of digital evidence. And oftentimes, that digital evidence is held by providers that are located across international borders or even if it's held by a locally-based provider, the data itself is held in a place that's across international borders.
And that raises all of the kinds of questions and challenges that we've been talking about today. And so there's an ongoing effort, and you can see this in the fact that the U.S. passed the Cloud Act, in the discussions in the EU and some of the discussions as part of the Budapest Convention. Efforts by governments to try and sort this out and figure out how to tie their jurisdictional rules to their sovereign interests in a way that makes sense.
And as we go forward and think about this, I think there's a few key points here. One is that obviously governments have a sovereign interest in preventing and investigating and prosecuting crime. Sovereigns also have a sovereign interest in protecting their citizens and residents and establishing the rules governing access to data of their citizens and residents in accordance with their normative values with regards to data protection and the relative balance of privacy and some of the free speech issues we raised already.
So the goal here is to come up with rules that protect and serve those sovereign interests. And we need rules that are practical, that are effective, and they're enforceable. I think we've heard and we'll probably continue to hear about the ways in which the mutual legal assistance process is often not efficient and often takes extraordinary long times for the processing of data. Even in situations where again, one government is seeking access to data that's solely sought with the investigation of local crimes.
And we need rules that ensure and promote civil liberties, privacy and respect for the rule of law. I'm happy to talk about how I think those map on to the various initiatives. But in the interest of time, I'll wait for the next round.
>> MODERATOR: Thank you very much, Jennifer. Bertrand, the Internet & Jurisdiction Policy Network has for some years now gathered stakeholders to try to come to solutions -- well, first of all to agree on the problems and then to move on to solutions. Can you tell us a little bit about that and the work ahead for the network?
>> BERTRAND DE LA CHAPELLE: Thank you very much. Hello, everyone. I will build on what has been said before. And most of the message is contained in this slide.
We're in a situation where yes, evidence is increasingly digital, yes, it applies to all types of crimes, yes it is often stored somewhere else and requires cross-border access. And in that context, both the governments and the private actors have developed ways to deal with how those requests for access to user information, in criminal investigations, are issued, transmitted and handled.
And they have done so both in the dimension of regimes, rules of various sorts. But also technical systems for sending those requests. Preparing those requests. Dealing and handling them. And this is a context that calls fundamentally -- and this is the one message I want to share here, for the notion of interoperability. Because without that, we have the danger of seeing the proliferation of various initiatives with all the best intentions, that may actually create additional conflict of laws if they are not coordinated.
And also, make the system difficult to scale. Because we're talking about numbers that are increasing at an enormous speed. The estimations vary, but people are talking now about rates of growth of requests for about 40% a year. And this is going to be more and more needed, and we may have to handle millions of requests and ensure, as Jennifer mentioned, sufficient protections for those.
The point that I want to highlight and the work that Internet and jurisdiction as a policy network is helping the different actors to do is to see how this interoperability can be framed into a certain number of shared policy standards and how we can bring the different categories of actors together. And we facilitate the work among more than 300 different entities, and particularly in one contact group. Some of the members are actually on this table.
To identify norms, operational norms, criteria, and mechanisms to foster this interoperability. And the two dimensions to finish are the ones I mentioned at the beginning. One is interoperability between norms, how to ensure that there is as little as possible, conflicts of laws and the principles and the mechanisms allow for the diversity of regimes. And at the same time, their coexistence. And the second dimension, which is often not addressed sufficiently, because we naturally first work at the level of principles. The second dimension is how do the technical systems and how will the technical systems that both the governments, public authorities, on the one hand, and the companies, on the other hand, are developing how will they interoperate.
Particularly, strong issue is the format for requests. How to make sure that there are interoperable, a bit like HTML or HTTP protocol allowed web pages to be understood irrespective of how they're structured. Likewise, the formats for requests potentially can be developed with a system of tags so that the systems for transmission can be jointly developed to be working together.
I don't get into details. I encourage you to do two things. One, to go on our site to download the operational approaches that have been produced by the contact group we had last year in 2018-2019. Which proposed a certain number of operational norms criteria and mechanisms for this. And the second thing is to invite you to come tomorrow at 1:15 to the session that we have for ING.
>> MODERATOR: Thank you very much Bertrand. Now let's turn our attention to the great work that the Council of Europe has been doing so far over the past two years in trying to come to legal normative solutions for at least a signatory of the Budapest Convention. Alexander Seger, if you could tell us a bit about what it is you've been doing so far and what's the latest development in coming up with a second protocol to the Budapest Convention.
>> ALEXANDER SEGER: This is the second time we do it in the same, which some may find boring, but it also allows us to track progress over time so we don't repeat the same things. I am happy to report we have had some progress in this field since we did the first one some time ago.
I'm talking about the work on the second additional product to the Budapest cybercrime on the way. This also helps when we talk about the threat to government access to data and so on, we have to keep in mind for cybercrime, less than 1% is reported to criminal justice authority. Out of the 1% that is reported less than 1% leads to a conclusive criminal justice outcome. And 0.01%, and that is an optimistic scenario. It may very well be 0.001%. What we also see, because criminal justice is not considered effective, power shift to the national security arena, as we see in the field of terrorism.
We talk here in the connection with the Budapest Convention, about a criminal justice approach, which means we talk about specified data that may be needed in a specific criminal investigation. It has nothing to do with bulk interception of data. Specific data is needed in a specific criminal investigation. This is very important to keep in mind.
And the power to obtain such data are based on law and have rule of law safeguards. The Budapest Convention cybercrime is a criminal justice treaty that follows this logic.
The work on the protocols. There were many, many years of preparatory work, but then formally the negotiations started in September 2017 and hopefully will be concluded in a year from now.
There are four blocks of elements that are under consideration. The first is to make mutual legal assistance process more efficient. Nobody proposes to do away with the MLA system. The MLS system is there, proven to function and in a good number of cases, it has protections built in. But we have to make it much more efficient and there are a number of articles that are currently being prepared or that have been already published, draft versions on MLA, more efficient MLA.
Secondly, we also need direct cooperation with providers in other jurisdictions. Google said already how many requests they receive. If the U.S. Department of Justice has difficulties to deal with 25,000 requests, on MLA and criminal methods, how could they deal with another 500,000 requests directed to service providers? The system would crash even more.
(Laughter)
We also need to think about situations where governments access data direct in other jurisdiction. Arrest a drug trafficker on the streets here, the system is open. Can you access the Gmail account because now you may access a server in another jurisdiction.
If we can agree on that, where do we stop? Where is the limit where we say we cannot allow government tracking? I am not sure that provision will eventually fly, but we have indeed many unilateral solutions from different countries. If we have such strong proposals we have to make sure we have strong data protection and rule of law safeguards built in. You can find online and we had last week some public hearings in Strasbourg. We have number public and draft versions. One on languages of requests because that's a major obstacle for MLA. One on video conferencing so you don't have to fly witnesses and experts back and forth. You can also have hearings by virtual conference. We have emergency assistance, 24/7 availability of MLA authorities so you can also get access to data via MLA on a weekend. In emergency situation, we have provision on direct orders to service providers of other parties for the disclosure of subscriber information. Not any data, subscriber information only.
And then we have an article that has been published which talks about giving effect to orders from another party. That is government to government. But it reduces the heavy machinery of MLA. There are a fewer levels that are cut out, but the protections are still there. This is what is currently on the table. Thank you.
>> MODERATOR: Thank you very much, Alexander. This was very interesting, indeed. I think we can proceed to a formal discussion among the panelists. I would start with perhaps one thing that wasn't mentioned in the slides of Jennifer around the ongoing developments at international or bilateral level to try and come to common solutions. And that is efforts from UN level to come to a treaty, a UN-level treaty on law enforcement level access to data.
It's an open question to the panel. We should specify that the resolution was proposed by Russia, Syria, North Korea and other countries. The resolution passed in the UN I think last week.
But obviously, this would just be the start of a very long process. All the legwork still needs to be done. Paolo, you have the Council of Europe already doing a lot of work in this space. How do you envision these UN level efforts? Is the UN the right forum for dealing with specific issue? I don't know who wants to take the question, first.
>> Let me start on this complicated question. Yes the UN would be the right forum for an international treaty on cybercrime. It would have been the right forum 30 years ago when the issue was discussed. I've been in that business quite a while. 30 years ago it was discussed with in Cuba, and there was no consensus on UN treaty on cybercrime. And the Council of Europe moved ahead and had already started before and moved ahead.
There is still no consensus today. If there were consensus today, yes, the UN should probably do that. But if there's no consensus, which is the case, it's counterproductive. Because it may lead in a further polarization of the world. And many of you may be aware of what happened in 2012 in Dubai, 102 Member States of United Nations are not party to that. The world is split and that will not move in either direction.
And the same may happen if now, without a consensus, were to go ahead in New York. And there's another concerning fact to that, that this is actually not really related to corporation and cybercrime or even electronic evidence. This is a purely foreign policy consideration, it has to do with who controls the Internet and how do states control the Internet. That's why so many oppose it, because they say we stand for a free and open Internet. This may again be yet another step in state control over the Internet.
Those are some of the concerns and also why it's been taken away from Vienna where normally criminal justice matters are discussed in the United Nations. There's 40 to 50 years of discussions on criminal matters taken to New York to take it into foreign policy, the diplomatic arena where it's discussed by diplomats.
>> Well, Brazil has always supported the discussions in the arena of UN but last week when the proposal was launched, Brazil has abstained. And I cannot talk for the Foreign Affairs, but from the perspective of the prosecution service, we've been supporting the idea of joining the Budapest Convention, because it's the framework, a Convention that already is there and is working. So we have present needs, and we have to attend it. We have to solve the problems that are already happening.
So that's our perspective.
>> MODERATOR: Bertrand, please go ahead.
>> BERTRAND DE LA CHAPELLE: I think it might be useful for the people in the room to understand that the different regimes that are being discussed and that Jennifer had mentioned. The Budapest Convention, (?) protocol, Cloud Act, and the evidence regulation in Europe.
They have sort of different architecture. The Cloud Act is something that is series of -- its second part is based on a series of bilateral agreements. Sort of a hub and spoke system. Whereas the United States evaluates the legal regime or system of a certain country and decides to lift under certain circumstances and conditions.
The blocking statute that prevents American companies from voluntarily disclosing information to a foreign investigation. At the moment, there is one bilateral agreement that has been drafted and published between the United Kingdom and the U.S. There was the announcement that Australia is developing an agreement as well. And they're probably are going to be others. The European Union as a whole is negotiating an agreement with the United States, although, the European Union considers that it is not really an agreement under the Cloud Act, whereas the United States agrees it is under the Cloud Act. Nevertheless, this is an architecture that is one country that has a major position because of the location of the operators, that would allow bilateral arrangements. The European Union eEvidence regulation currently in front of the European parliament, on the proposal of the European commission by the council is a mechanism establishing the capacity for European Union law enforcement to issue binding orders to operators who provide services in the territory of the European Union. That being said, there might be situation where's those binding orders would be in contradiction with the blocking statutes such as the one that currently exists in the U.S.
That would prevent the operators for complying with this order. So it's a different approach, which is not the country that has main operators granting access to a certain number of countries with sufficient standards, in its view. It is a block of the European Union saying we should have the capacity to impose and issue a compelling order.
The Budapest Convention, to go quickly, is on the basis of the actors who already participate in the Convention, which is more than 60 parties. And it is adding a layer, but what is interesting is that for each of those different things, there is the question of scalability. How does the regime grow afterwards and how does it expand. And the Budapest Convention was an example of starting with a small number of actors and growing afterwards.
To finish on the question you asked regarding the UN, there's the whole question of is the goal to have something that is universal in the end, or should we start with universal? And the problem of consensus that Alexander is raising, is typically the problem. Because even in the UN, even if you start a negotiation, there is currently not sufficient agreement on what could be achieved. Even within the Budapest Convention negotiations, it is only on a certain number of issues that consensus begins to be achievable.
So I think it's important in this discussion to understand the different dynamics between the four possible approaches and their likelihood of success.
>> MODERATOR: Thank you, Alexander. You wanted to add a word to that?
>> Just one point of where it stands in the UN. This was a resolution brought in by seven, eight countries. Russia, China, North Korea, Venezuela, Cambodia, I believe and some others. It was voted in the third committee of the General Assembly of the UN about two weeks ago, and it will still have to go to the plenary of the UN General Assembly. This is still not done. Some countries may say we support it or don't support it. The vote will be in mid-December. If it goes through it will establish a committee for New York for next year onwards.
>> OK. Thank you. Perhaps we can focus a little bit of our attention to something that's been talked about a lot lately, which is the U.S. Cloud Act.
Jennifer, you've researched extensively about what the Cloud Act does and doesn't do. There's probably a lot of misconceptions about what is national legislation, what the brings forward, the advantages and pros and cons of this legislation. Can you shed some light on that, please?
>> JENNIFER DASKAL: Sure. Thank you. As probably all of you know, the Cloud Act has two very distinct parts. They're interconnected, obviously. They're both dealing with access to data across borders, but they're very different. We've talked a bit about the second part already, that is the response of the U.S. Government to the kinds of frustrations that we heard from Brazil about foreign governments seeking access to data that's held by U.S.-based service providers. And even in the investigation of local crime, according to local rules and being told by the service providers that they can't disclose at least the communications content. Meaning the substance of messages, et cetera.
The U.S. Government insists ultimately that a U.S. prosecutor gets a U.S. warrant according to U.S. legal standards, probable cause standard in terms of standard for accessing the data, but also incorporates a number of other U.S. standards including the first amendment, free speech standards of the United States, which are, as everyone here knows, not the same as free speech standards elsewhere.
So that has caused enormous frustration for foreign governments. So one piece of the Cloud Act authorizes the U.S. Government to enter into executive agreements with foreign partners in order to lift that blocking statute as Bertrand said, and enable those foreign partners to make direct requests to U.S.-based providers.
This part of the legislation includes a number of criteria that the foreign partner has to meet. So before an agreement can even be entered into, the partner has to be certified by the U.S. as being rule of law and Human Rights compliant, and each and every request has to meet a number of specified standards, including it has to be targeted, subject to some sort of review or oversight. It has to basically protect free speech principles, although it doesn't insist exactly on the U.S. first amendment standards. So that does allow for some more flexibility than currently exists in the law.
And a whole host of other requirements. And importantly, and this goes back to the question of how do you draft these kinds of agreements that protect sovereign interests. It only authorizes foreign governments to make these direct requests for data of foreigners outside the United States if the foreign partner is going to request the data of a U.S. citizen or U.S. resident, it continues to need to apply through the mutual legal assistance process, based on the theory that U.S. interests are tied to -- in this respect, are tied to its citizens and residents and U.S. standards and U.S. rules should apply to the gathering of data of U.S. residents. These agreements have to be reciprocal. For example, the U.S. entered into an agreement with Brazil, the U.S. Government could not use this agreement to make direct requests for Brazilian residents and data. They would have to use the mutual process, reflecting the idea that Brazil should set the standards for access to residents and citizen data.
What's probably been most controversial is what I call the first part of the Cloud Act, which is a response to a lawsuit that was brought by Microsoft -- not a lawsuit, but a refusal to -- a motion to quash a warrant that was issued by the U.S. Government. And Microsoft moved to quash the warrant on the grounds that the U.S. Government was seeking data pursue want to a warrant issued by a judge based on the standard of probable cause from Microsoft.
But the data that the U.S. Government sought happened to be stored in Ireland. And Microsoft said your warrant authority doesn't extend to data that's outside the territorial boundaries of the United States. The dispute went all the way up to the Supreme Court. After there were arguments before the Supreme Court and they could issue the ruling, the Congress passed the Cloud Act, and that clarifies consistent with the U.S. Government's position that the location of data does not delimit access. If the U.S. Government goes to a court and gets a warrant based on probable cause that criminal investigation -- to be clear we're talking about criminal issues, and issues that warrant on a U.S. based service provider or another service provider over which it has jurisdiction, that provider is obliged to turn over the data within its possession, custody or control, regardless of where the data happens to be located.
One important thing to understand about this is this is the long standing practice on the assumption of I think most parties in the system up until the point and time in which Microsoft issued this challenge. It had always been the case, at least in the context of another form of compelled disclosure orders in the U.S. subpoenas when the U.S. issued subpoenas on those companies they are to turn over material without regard to location.
Another piece of this is again to remember that at least when we're talking about content there's an obligation to get a warrant which requires review by an independent judge that in fact that there is a predicate criminal investigation that the evidence, that there's probable cause to believe that the evidence is necessary for that investigation, and that the request is particularized and targeted in accordance with U.S. law requirements.
And then one -- I guess I'll – oh, one last piece worth emphasizing about the first part of the Cloud Act is that it also incorporates new provisions that hadn't formally been in place in U.S. law. That explicitly provides a mechanism for providers to move to quash if the U.S. Government is seeking the data of a foreigner outside the United States and if the request creates a conflict with U.S. law.
And there's two separate ways that providers can do that. One is via this new mechanism that was created by the Cloud Act, although it only can be triggered in very limited and currently nonexistent situation. But also by explicitly emphasizing the availability of what's known as common law claims. To hammer again the point that this reflects the idea that foreign governments should and do have a say in how other governments, including the U.S. Government, access the data of their own citizens and residents.
But it doesn't make sense for, say, Ireland's laws to restrict the U.S. Government when it's trying to access the data of its own citizens, its own residents in the investigation of local crime just because the data happens to be stored in Ireland.
>> MODERATOR: Thank you very much, Jennifer. Fernanda, maybe you wanted to react to this?
>> FERNANDA TEIXEIRA SOUZA DOMINGOS: I want to make a comment. I think -- well, the question here is that the approach of United States is that they have jurisdiction over the data if the companies have their headquarters there. If they're American companies.
And what Brazil's legislation and evidence proposals tell about that jurisdiction exists if the services are being provided in the territory.
So the thread of thought is that if a company is doing business in my country, it has to follow my laws. And also, if a crime is being committed in my territory by a citizen of my country -- I mean nothing is about -- (?) only the data, that we don't know where it is. We have to be able to make this law to be complied by the companies.
And when it comes to the -- for instance, incitement to racism, which is very important theme in Brazil. Or denial of Holocaust, that was already judged also in Supreme Court. If I had crimes like that, and if I needed content from a company in the United States, I wouldn't get it, even via MLA. And I don't think the Cloud Act is going to solve it.
That's -- I don't know. We'll have to address it sometime. I don't know.
>> MODERATOR: Yes, please go ahead.
>> JENNIFER DASKAL: A couple things. As Bertrand pointed out there's only been one executive agreement under the Cloud Act so far. It was enacted 18 months ago and we have seen one executive agreement. There's some discussions there's negotiations with Australia as well. But as Bertrand, this is a slow process, and not something that is likely to solve all of the problems immediately. The second part of the Cloud Act pursuant to which executive agreements could be entered into is meant to deal with exactly, precisely the problems that you've identified. Whether it's the right or sufficiently -- whether it's going to be effective and to what extent and how many countries ultimately enter into these agreements and get the kind of access that's being discussed here, is I think an open question.
So it's a limited solution for now.
>> MODERATOR: Great. Thank you very much. I think we can proceed with questions from the audience here. But also online. Rachael, please wave a hand if you have anything coming from the audience and we'll take the questions.
Just before we start, a couple of things. Jennifer has to take a flight rather early, so leaving five or ten minutes before the Q&A session ends. So please bear that in your mind. As you ask your questions, please introduce yourself and the name of the organization you represent. Thank you. Are there any questions? Please, the lady at the back there.
>> Hi, I teach at a university in Bangladesh and the question I have in discussions here, the -- I mean multiple panelists raise, talks about the criminal justice system and the specific criminal acts as if it's neutral and specific.
But the problem with criminalization and criminal justice system is not neutral. For example, when you're talking about kind of data sharing for specific criminal acts or criminal activity, what is your reaction kind of when government -- you know, go toward large-scale criminalization of activities like sex workers. Drug use. And that actually ends up negatively affecting large sections of society who are already marginalized. And in these kind of discussions, they actually have no participation.
So all these kind of criminalization, criminal justice processes, there's no representation or very little representation of people who are being criminalized.
>> MODERATOR: Thank you. Anybody want to take that question first?
(Laughter)
>> OK. The point is that there is globally some sort of consensus of what is to be criminalized. And that would reflect more or less the articles 2-11 of the Budapest Convention in terms of cybercrime. There is a lot of dis agreement for many other things. There is a big risk that we also see that countries adopt domestic substandard criminal law, meaning criminalizing conduct in a very broad way, that would leave a lot of room for interpretation that has no clear way, no clear limits. That leads to cybercrime investigations and -- for free speech and sex work and so on. When it comes to the international corporation part of that, the dual criminality provisions usually kick in including in the forms of the direct disclosure we published a few weeks ago. There you would not get cooperation from another country, on pornography in general. They criminalize child pornography. So you get child abuse materials online, but in many cases you do not get cooperation on forms that are not criminalized in both countries.
But again, that does not help you nationally in your --
>> MODERATOR: Just one second.
>> I would have to disagree that there is broad consensus globally of what should be criminal and criminalized behavior. That is part of the problem, people who are being criminalized generally are not part of these discussions. For example, sex workers, right?
I would again, strongly disagree with that contention that there is broad agreement.
>> Sorry, I didn't say that. I said there is a range of offenses that are broadly agreed upon. And they would -- one second. There is 100-plus countries that have implemented, whether they're parties to the Budapest Convention, they've -- article 11, data access is and interference and child pornography. There is agreement on that. For anything else, there's not necessarily agreement.
And there's certainly no agreement on -- and if you want to kill any discussion about something you talk about legalizing or criminalizing prostitution, then immediately you kill the agreement and you will not go ahead. There's no agreement on that. That's why you don't find international treaty, multilateral treaty, which talks about criminalizing sex work or prostitution. That's what I said. So I didn't say there's agreement on that.
>> Bertrand, just one second. I think this points to the perhaps, the discussion around precisely that effort, the UN effort, to get some sort of general consensus.
I mean over the past decades, there hasn't been any consensus at the UN level. And probably the definition of crimes is probably one stumbling block, and I think bit by bit, moving forward, the international community can slowly build a framework that would work.
>> BERTRAND DE LA CHAPELLE: I would like to intervene in that regard, because as we said in the presentations before, we're not only talking about cybercrime, but we're talking about also, a lot of criminal investigations that increasingly require access to electronic evidence.
So in as much as there is a relative convergence on the definitions of cybercrime in the context of the Budapest Convention, for instance, of course, there's an enormous disparity on what is criminalized in one country or the other.
I want to highlight that one of the things that needs to be done in those debates, and that's one of the real focus of what we're trying to accomplish, is to allow the different actors to formulate the problem in a way that is a common formulation. And I think what we all are trying to do -- and when I say all, it is the law enforcement, the different parts of governments, the companies, the Civil Society actors, international organizations.
What is at stake is what are the substantive and procedural guarantees under which a regime or regimes, plural, would allow law enforcement or investigators in one country to request directly from a foreign operator, information about a user. And this notion of procedural and substantive guarantees goes directly into your direction, because one of the fundamental stumbling blocks and difficult things is do you accredit one country's legal system or not. Which is a little bit what the Cloud Act intends to do. Do you evaluate each request case by case?
Do you accredit a corporation mechanism between two countries on some topics of criminality? Do you use the approach that the European Union evidence regulation and -- which is to set a certain bar of penalties and saying only serious crimes of more than three years -- and actually when it is now in front of parliament, the European parliament says the bar at three years is too low. It should be five years.
So how the nuances of those things can be taken into account so that the regimes can be efficient and at the same time, not cover absolutely any crime, is one of the big challenges under consideration today.
And I want just to highlight that in one of the other groups that we facilitate, there is a distinction emerging which is the notion of international normative consistency.
And very rapidly, you have things where everybody agrees are either behavior or content that is inappropriate and the standards are relatively uniform. Or those everybody have to be addressed, but the standards vary.
Other situations where there's disagreement among the countries among whether something should be criminalized or not. The challenge in building the regimes is to take into effect the commonality so the protections are -- at the same time the system is efficient. It cannot address in an international regime, the participation of the people in the national legislation that might impact them. That's another issue that cannot be solved at the international level, and I think you have a point.
>> MODERATOR: You wanted to raise a point?
>> LUDMILA GEORGIEVA: Maybe also to your question, I think we all agree there is no redefinition of criminal offenses. And this is probably -- it's about procedural and substantial safeguards. We face in the European Union, what is a crime and what is not a crime. So kind of a common notions definitions of what is supposed to be part of the evidence proposal would be useful, exactly for this. Because honestly, I don't think that anybody wants data provider to be the judge of what is a crime or not. I think this is a point where there is kind of the question of safeguards in place in order to balance case by case, but also the question of some legal framework that allows everybody to decide and say OK, this is it, and I can just -- this requests, or response to this crime. This is the framework. These are the safeguards. OK, and then we can disclose, because to disclose the data, and this is again where we're trying to emphasize. It's about law enforcement, but it's also about disclosing users' data, which could be in some cases very, very tricky, especially when it comes to journalists or protected groups. I think this is the question where I agree with Bertrand about procedural stuff, but also kind of the notion of what is respect. And that's why international agreements are so useful because they tackle exactly this kind of issue, and then they can agree on specific definitions or notions.
>> MODERATOR: Thank you. Just one thing. If you wish to ask questions, please proceed to either of the mics on either side of the room, please. I see people are already queuing. So please go ahead with the questions.
>> Thank you. That was a very interesting and enlightening panel. Thank you for that. I'm from DiploFoundation. Several speakers -- to have access to users’ information. There is a directive that is approved but has not been transcribed into international laws on the protection of personal data in the context of investigations in Europe.
If I'm not mistaken the commission has referred Greece and Spain to the court for not transposing this directive. So could you shed some light on the importance of this directive and how it fits into the debate of what you're talking about and just a quick question to Jennifer. What I understand from what Fernanda said is from the standpoint of the governments that fears that they have jurisdiction over a criminal case in the first place. It feels like a quick fix to have in the Cloud Act, provisions that would make the U.S. Government be in the position to judge, if the country should have access to that information or not. Even though this could be a partial solution to the problem, would that be a political solution to the problem in the long one? Because I am afraid that many countries will not be judged to be reliable enough to have access to that information under the Cloud Act. Thank you.
>> MODERATOR: Jennifer, maybe you want to start?
>> JENNIFER DASKAL: Sure. Just as background, the Cloud Act is meant to alleviate, not exacerbate the situation that we're faced in, which is exactly what was presented. Is that there are times in which two countries claim different jurisdiction in some sense or another, over the same company.
So Brazil may say Google is offering services in my country, and therefore is subject to my law. Which I agree is a completely legitimate exercise of jurisdiction, depending on the circumstances. And Brazil may say under my law, Google has to turn over a certain amount of communications content. Google, that is also subject to U.S. law, is prohibited under U.S. law from turning over that communications content. Not based on the Cloud Act, based on an old 1980s act that was enacted before there was a cloud and anyone was thinking about these types of issues.
And so the Cloud Act is meant in certain circumstances to alleviate that, to lift the blocking statute so that Google or any other company could turn over that data without violating U.S. law.
The problem facing many countries is that the mechanism for lifting that blocking statute is relatively limited, and only kicks in after a case by case assessment of the countries and the entering into this executive agreement of which there's none that are yet implemented. There's one that's been drafted and now there's 180 day waiting period in Congress.
>> MODERATOR: Please go ahead.
>> FERNANDA TEIXEIRA SOUZA DOMINGOS: I want to talk about the police directive, OK?
>> MODERATOR: Please go ahead.
>> FERNANDA TEIXEIRA SOUZA DOMINGOS: I believe you were referring to the police directive of EU. What I can say from the perspective of Brazil, besides the safeguards, I have already told that all data is disclosed only with judicial order, and it also -- everything that is collected in the investigation is disclosed and can be challenged in court and also can be redressed if there has been any abuse.
And in Brazil, we have a law that has already passed, but it's not enforced yet, which is the data protection law. Very similar to the GDPR. And there's an article there that refers to all data that has to be regulated for investigation and security. And we'll have to write something like this police directive.
That's what I can add to the panel here.
>> LUDMILA GEORGIEVA: Maybe just a bit of my previous life when I was on the lawmaker side. The GDPR and -- justice directive is one complex.
The policy and justice directive is meant to be actually for processing data for purposes of investigating, deterring and protecting crime and so on. But it's meant to be very narrow so that only specific types of crimes and law enforcement authorities with this kind of purposes under the directive have this a bit more privileged regime. Otherwise it's GDPR to apply for everybody. And the transposition framework of timeline passed into some countries didn't implement it on time. That's why the commission is going for after this countries. So this is for the explanation of how that explains a bit. On the question of jurisdiction. I think for us, been referencing all the time here, it's kind of the thing the Budapest Convention, the article 18 has a very good notion of jurisdiction. We agreed that the localization of data is not the one we should reference to, but article 18 of the Budapest Convention is something that has set out several requirements. For example, the (?) powers and controls. I think this is a very important element because it's not like Google everywhere has the data -- entity of Google has access to all of the data which are required. Honestly, it is a very sensitive issue where specific team is trained to deal with all these requests, and that's why not all of our entities are able to do this or we don't want them to do this, because this is a very, very specific and sensitive issue, which we want to have only trained people to deal with it. I hope that gives a bit of our enlightenment. As I said, Budapest Convention article 18, we think that the eEvidence, the regulations should follow this provision.
>> I'm very happy that you like article 18. In the interpretation of the guidance we adopted two years ago, I think that's a very important concept. Just on data protection. It's extremely important and extremely complicated.
And what adds to the complication is that we talk here about asymmetric incorporation, one country sends request to another country. This request already contains personal data. If the service responds to that, personal data is transmitted. How do we sort this out?
Probably the most complicated one to negotiate, it's very, very hard. But we're making good progress there, and hopefully in January or February we have a draft provision, and then maybe we can also go out and have some more consultation with data protection experts on that. So it's -- new directive on data protection, the GDPR is very important. But we will try to build specific data protection requirements into the protocol itself.
>> Thank you. Rachel do we have any questions from the online audience? Fantastic. We can proceed from the audience.
>> Elizabeth, and I'm a member of some nonprofit organization on trying to cooperate to reduce existence risks. So inclusion. Governance inclusion. Reduce inequalities and mitigated risks. So violence of abuses. I would like to ask so what about information governance. Because of course, there are different kind of access to information for different reasons. Sometimes also because of security or because of interests.
What happens if institutions are abusing their powers against citizens. For example, what about targeting campaigns? What is possible to do to declare these abuses and get access to justice information and the dignity.
So it's incredible what we have achieved with the research. We have mind control technologies. So an incredible amount of -- so advanced technologies. What about (?) and the use of it? Why there is lack of information? And so what happens if technology is used to repress, like, civil freedom or to manipulate human behavior? What about identity? What happens with human dignity. What is identity now? Thank you very much.
>> MODERATOR: Thank you. So if I were to rephrase the question very shortly, it's to what extent the solution -- what we've been discussing, on providing some kind of procedural substantive safeguards to prevent states from abusing their investigative powers. Particularly in state where the rule of law may not be something that they particularly follow. They have their own system of law.
>> Maybe a short response from me and others who want to take it up. In 2008 there was an important decision in the European court of Human Rights. A case of KU versus Finland. It's about the human dignity. It's a dignity of a 12-year-old boy from Finland.
And on a social dating website, his details, his picture, his telephone number and so on were posted with him offering sexual services.
And at that time, the laws protecting privacy in Finland were so strong that the service provider under no circumstances was permitted to disclose who posted this, right? The law was an absolute prohibition of that. And this went through the courts in Finland and the court of Human Rights in Strasbourg, and the court of Human Rights made the remarkable decision saying that Finland was responsible for not putting the laws in place to protect the dignity of this boy in this case. And that governments have the obligation, the positive obligation, to protect individuals that are victims of crime. Also through criminal law.
Including article 18 of the Budapest Convention which is specifically mentioned in that decision by the way.
So we have to look at it also from that angle, that if we have 100,000 cybercrimes, and quite a number of them violating the dignity of individuals and only 0.01% of them prosecuted, we have a failure of protection of dignity. Just one moment. It's getting longer and longer my answer here.
Last week in the conference, we had people from the prosecution office of Buenos Aires. That office receives every month, on average this year, the first ten month, they receive 30,000, that makes 3,000 pieces of information from Internet center for missing and exploited children. 3,000IP addresses pointing at child abuse materials being disseminated downloaded and shared and so on. That's just that office per month.
And the difficulty to then go and get, find out who are the subscribers, are enormous. That's also, we have to keep in mind if we talk about human dignity. So to protect the human dignity of lots of children whose pictures are being exploited and they are being exploited and being shared. That's also protecting human digital. Thank you.
>> MODERATOR: Well, I think we can proceed to the next question.
>> Thank you so much. I work for the Swiss federal department for Foreign Affairs. And my colleagues from law enforcement also say that the issue is growing day by day and it's getting more and more important to combat (?) crime and also organized crime. My question is more of a strategic nature.
We are obviously members of the Budapest Convention, and we would like all the world to join. But as we have also discussed, we see at the global level the resolution of the third committee of the UN and others that have considered joining the Budapest Convention are cosignatories, co-sponsors of this resolution.
So where do you see this going at a global level? Do we have some elements for agreement on combating cybercrime on digital evidence globally, or do you rather see what was mentioned in the beginning. Kind of a schism globally with regard to this question.
So thanks for giving us a bit of your version where we are moving globally. Thank you.
>> MODERATOR: Thank you. Maybe we can also take the next question, and then we'll address them at the same time.
>> My question is specifically directed to Alexander. It is what Alexander mentioned about 0.01% of the consequences being brought. From the context or perspective of Internet user, especially who's a victim of crime. Or (?) of law enforcement (?). How optimistic are you in terms of global arrangement or agreement for combating cybercrime whether it is Budapest Convention or the UN Convention if it ever comes. Or a matrix of different interoperable agreements which can take care of bringing consequences to all cyber criminals? Thank you.
>> MODERATOR: Thank you.
>> ALEXANDER SEGER: Maybe I'll start very briefly. There is no single treaty that can solve all the problems. That's absolutely clear. That would be completely surrealistic. I think we need a matrix of multiple solutions that includes what (?) already has. We have a partnership within the (?). It includes what many other organizations are doing. But what I'm afraid of and what I see is that the criminal justice response, which is a very protective response, right? Because it protects the rights of individuals.
That's what criminal procedure law is about. It empowers law enforcement to do something with conditions, and then the remedies. Criminal response is a very protective response. The (?) response is completely sidelined and we have to make it more effective. And not leave it only to the national security arena where the protections aren't necessarily there, but there's a strong interest in protecting critical infrastructure. But the interest and rights of victims are also sidelined. This is coming to that question.
Regarding the question on the U.N. general assembly, how this is going. At the moment, the risk of polarization is really there. And it's not something we now make up. We have this experience of the 2012 international telecom regulations. The big question is do countries that are opposing that resolution for exactly this reason and some other reasons, should they engage or should they stay out?
That is an open question. There's no answer to that. It was heavily discussed last week in connection with the (?) conference. There's an interesting video of a presentation by a representative of the Russian federation and then some responses to that.
But it's an open question whether to engage or not engage. In principle, I think one should engage. But it also depends the conditions. Everyone is taken for a ride and there's a -- this has to be evaluated from government around the world very carefully. Certainly not up to me to decide.
>> JENNIFER DASKAL: So two just quick thoughts on the question of are we headed towards a schism. And one, just to build off on what was just said. To the extent that we're building data regimes that facilitate law enforcement access to data, it is as I think we've all highlighted, important for those regimes to include accountability provisions and substantive and procedural safeguards. And I think we need to be weary of initiatives that don't include that as one of their primary key elements.
Secondly, on the schism piece, I think we're headed towards a schism, and I think in part because at the same time that the initiatives that we've highlighted here with respect to eEvidence, to some extent the Cloud Act, and certainly the Budapest Convention negotiations are all geared at facilitating cross-border access to data in a world in which the parties acknowledge and accept that data moves around with relative ease and that the sovereign interests at stake are not necessarily linked to the location of the data.
There is a very different approach, a data sovereignist approach, and people use that in word in all ways. I would link to some of Russian's and China's initiatives in which the territorial data sovereign link is very intact and there's a real concern about any effort to try to access data across borders. And more importantly a real move towards data localization as a means of securing and ensuring control.
And to the extent -- those two visions are just not consistent. So unless we figure out a way to kind of mediate those two visions, I think we increasingly get into a world in which we have competing systems to some extent.
>> BERTRAND DE LA CHAPELLE: As said, it triggers the reaction. Somebody in my team is the challenge is to bring people to justice, not -- integrating the notion of the user and how the rights are protected is a very important element. I want to highlight as a sort of conclusion that time is of the essence. We're under pressure at the moment because of the scale of the problem that is growing and has been repeatedly mentioned. This requires technical streamlining, engineering of the relationships, but it requires frameworks.
We absolutely need frameworks. And if we don't, a certain number of consequences. One, there will be continuation of things that are without guarantee. And the companies will be under pressure. They will have to accept the request without any framework that is really clear and guarantees the right to appeal and so on.
Second, as Jennifer mentioned, as long as we do not have a framework that establishes clearly the conditions under which a country can conduct investigations and access information that is held by another company outside of their frontiers. The pressure, rightly or wrongly can be an alibi to invoke that investigation.
On the other side of the coin, the countries that have the power will exercise increasing (?) authority. When you think about that this is not the way we are thinking about sovereignty in the classical way. And the challenge is how do we define sovereignty in a digital age and how does it really function. And I want to highlight one thing, which is that the current international system based on territorial jurisdictions is actually what prevents cooperation on those issues. It is what is hindering the capacity of the actors to work together. And even if the time pressure is important, I takes as a positive sign the fact that there is strong awareness. I would echo what Alexander was saying at the beginning. There is progress. It's not as quick as it should be, but there is progress, and I would like to quote one of my favorite philosophers who says pessimism is a matter of mood and optimism is a matter of will.
The framing components are there, but there is a need to go forward and if there is not enough desire to solve the problem, we can get into a protracted battle for five or ten years with two camps fighting against one another.
>> MODERATOR: Thank you so much. That concludes this workshop. I think just to Bertrand's -- did you want to say something? OK. Just to go back to Bertrand's point, I think this basically, this is quoting the idea of one world, one net and one vision. And it's about building that block by block. And it takes time, but we need to do it.
I think next year we'll have, again, plenty to talk about. So I thank all the panelists. Thank you very much. And thank you to the audience. Thank you to the online audience. And I look forward to seeing you next year.
(Applause)