GET request with utf-8 query parameter

[cwarny]

When using the Client class to build a request (through its request method) from a URL that contains query parameters, it seems like the only way to pass the query parameters to that method is by having them already urlencoded in the url (no params argument in the request method). The problem with that is that, in the constructor of the Request class (invoked by the request method of the Client class), the URL is converted to unicode. But if the URL contains utf8 characters, the Request constructor is converting urlencoded utf8 characters to unicode, assuming they are non-urlencoded string. The fact that the utf8 characters have been urlencoded makes the unicode conversion incorrect. In class Request's to_url method, you end up with incorrect query in the following line:

url = (scheme, netloc, path, params, urllib.urlencode(query, True), fragment)

Our use case is interacting with the Twitter API, using their search endpoint with a query in farsi language. The query gets corrupted because of the above process (I think, please confirm) and the Twitter API request returns incorrect results.

[jaitaiwan]

Thanks for bringing this to our attention. Do you have a suggested way of resolving this? We're happy to accept pull requests if you have a solution in mind.

[cwarny]

I'm going to think of a way and suggest it to you before starting to work on it

[jaitaiwan]

Hey @cwarny have you had any thoughts here?

[cwarny]

Hey. I ended up refactoring our codebase to use the requests library and it solved the problems we were having. It seemed like a more expedient solution for the tight deadlines we had. Sorry I didn't get a chance to work on a solution for this