Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sanitize http.url attribute #1961

Merged
merged 15 commits into from
Apr 8, 2021
Merged

Conversation

pellared
Copy link
Member

@pellared pellared commented Apr 6, 2021

Fixes #1884.

Changes

Please provide a brief description of the changes here.

For significant contributions please make sure you have completed the following items:

Constraints

  • For requests without username/password data the http.url value would remain the same. For ASP.NET instrumentation it is uri.ToString() for HTTP instrumentation it is uri.OriginalString. We can consider unifying it but rather in separate PR.
  • The public API is untouched. In theory, we could expose the method which sanitizes an URL in a separate PR.

@pellared pellared changed the title [WIP] Sanitize http.url [WIP] Sanitize http.url attribute Apr 6, 2021
@codecov
Copy link

codecov bot commented Apr 6, 2021

Codecov Report

Merging #1961 (8c5360e) into main (d073906) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1961      +/-   ##
==========================================
+ Coverage   83.54%   83.57%   +0.03%     
==========================================
  Files         192      192              
  Lines        6156     6162       +6     
==========================================
+ Hits         5143     5150       +7     
+ Misses       1013     1012       -1     
Impacted Files Coverage Δ
...umentation.AspNet/Implementation/HttpInListener.cs 88.88% <100.00%> (+0.31%) ⬆️
...tp/Implementation/HttpHandlerDiagnosticListener.cs 70.96% <100.00%> (ø)
...strumentation.Http/Implementation/HttpTagHelper.cs 93.54% <100.00%> (+0.69%) ⬆️
...ter.ZPages/Implementation/ZPagesActivityTracker.cs 100.00% <0.00%> (+2.85%) ⬆️

@pellared pellared changed the title [WIP] Sanitize http.url attribute Sanitize http.url attribute Apr 7, 2021
@pellared pellared marked this pull request as ready for review April 7, 2021 17:47
@pellared pellared requested review from a team and cijothomas April 7, 2021 17:47
Copy link
Member

@CodeBlanch CodeBlanch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

[InlineData("http://localhost/", "http://localhost/", 0, null, "TraceContext")]
[InlineData("http://localhost/", "http://localhost/", 0, null, "TraceContext", true)]
[InlineData("https://localhost/", "https://localhost/", 0, null, "TraceContext")]
[InlineData("https://localhost/", "https://user:pass@localhost/", 0, null, "TraceContext")] // Test URL sanitization
Copy link
Contributor

@utpilla utpilla Apr 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add test cases to cover all the parts that would make up the new Uri such as scheme, authority, path and query, fragment etc.?

How about adding two unit tests like these which would ensure that Uri.PathAndQuery (/Home/Index.html) and Uri.Fragment (#search) are concatenated?

Url: https://user:password@localhost:443/Home/Index.html#search
ExpectedUrl: https://localhost:443/Home/Index.html#search

And one for http
Url: http://user:password@localhost:80/Home/Index.html#search
ExpectedUrl: http://localhost:80/Home/Index.html#search

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did my best to address it here: 5f517db and here: 275d431

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment.

@pellared
Copy link
Member Author

pellared commented Apr 8, 2021

I noticed that there is a potentially unstable test:

  Failed OpenTelemetry.Instrumentation.Grpc.Tests.GrpcTests.GrpcAspNetCoreInstrumentationAddsCorrectAttributes(enableGrpcAspNetCoreSupport: True) [22 ms]
  Error Message:
   System.NullReferenceException : Object reference not set to an instance of an object.
  Stack Trace:
     at OpenTelemetry.Instrumentation.Grpc.Tests.GrpcTests.GrpcAspNetCoreInstrumentationAddsCorrectAttributes(Nullable`1 enableGrpcAspNetCoreSupport) in D:\a\opentelemetry-dotnet\opentelemetry-dotnet\test\OpenTelemetry.Instrumentation.Grpc.Tests\GrpcTests.server.cs:line 83

@pellared pellared requested a review from utpilla April 8, 2021 08:56
[InlineData("https://localhost/", "https://user:pass@localhost/", 0, null, "TraceContext")] // Test URL sanitization
[InlineData("http://localhost:443/", "http://localhost:443/", 0, null, "TraceContext")] // Test http over 443
[InlineData("https://localhost:80/", "https://localhost:80/", 0, null, "TraceContext")] // Test https over 80
[InlineData("https://localhost:80/Home/Index.htm?q1=v1&q2=v2#FragmentName", "https://localhost:80/Home/Index.htm?q1=v1&q2=v2#FragmentName", 0, null, "TraceContext")] // Test complex URL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could update this test case to use http instead of https to test that we return the right Uri.Scheme in the URL.

@utpilla
Copy link
Contributor

utpilla commented Apr 8, 2021

@pellared Could you please also add the logic to sanitize http.url to ASP.NET Core Instrumentation as well?

activity.SetTag(SemanticConventions.AttributeHttpUrl, GetUri(request));

@pellared
Copy link
Member Author

pellared commented Apr 8, 2021

@pellared Could you please also add the logic to sanitize http.url to ASP.NET Core Instrumentation as well?

activity.SetTag(SemanticConventions.AttributeHttpUrl, GetUri(request));

@utpilla I do not see this issue there. See:

private static string GetUri(HttpRequest request)
{
var builder = new StringBuilder();
builder.Append(request.Scheme).Append("://");
if (request.Host.HasValue)
{
builder.Append(request.Host.Value);
}
else
{
// HTTP 1.0 request with NO host header would result in empty Host.
// Use placeholder to avoid incorrect URL like "http:///"
builder.Append(UnknownHostName);
}
if (request.PathBase.HasValue)
{
builder.Append(request.PathBase.Value);
}
if (request.Path.HasValue)
{
builder.Append(request.Path.Value);
}
if (request.QueryString.HasValue)
{
builder.Append(request.QueryString);
}
return builder.ToString();
}

@utpilla
Copy link
Contributor

utpilla commented Apr 8, 2021

@pellared Could you please also add the logic to sanitize http.url to ASP.NET Core Instrumentation as well?

activity.SetTag(SemanticConventions.AttributeHttpUrl, GetUri(request));

@utpilla I do not see this issue there. See:

private static string GetUri(HttpRequest request)
{
var builder = new StringBuilder();
builder.Append(request.Scheme).Append("://");
if (request.Host.HasValue)
{
builder.Append(request.Host.Value);
}
else
{
// HTTP 1.0 request with NO host header would result in empty Host.
// Use placeholder to avoid incorrect URL like "http:///"
builder.Append(UnknownHostName);
}
if (request.PathBase.HasValue)
{
builder.Append(request.PathBase.Value);
}
if (request.Path.HasValue)
{
builder.Append(request.Path.Value);
}
if (request.QueryString.HasValue)
{
builder.Append(request.QueryString);
}
return builder.ToString();
}

Thanks @pellared for pointing this out. This issue does not exist for ASP.NET Core Instrumentation.

@cijothomas cijothomas merged commit 44cd88e into open-telemetry:main Apr 8, 2021
@pellared pellared deleted the sanitize-http.url branch April 8, 2021 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HTTP Span Attributes: http.url must not contain username / password
5 participants