The blogger has been a bookworm for nearly a decade, having read online novels since her high school days. Every day half a day to see the evening to see ah, finally eyes also myopia, grades also declined (… Seems to say far) recently chasing Chen Dong’s “holy ruins”, recently wrote the wonderful part, has been waiting for updates. But you can’t keep your browser open to refresh… As a result, we play the spirit of self-sufficiency programmers, write a program to monitor, and then update the time, automatically open the browser, and then you can enjoy reading.
– Open VS2017 and create a new… Any project, and say
HttpClient httpClient = new HttpClient();
var message = await httpClient.GetAsync(a);
var html = await message.Content.ReadAsStringAsync();
Copy the code– And then the customary Debug looks at the HTML value… Oh, my God…
What is this??
– Then I thought it was a coding problem, so I added more coding
var contentType = message.Content.Headers.ContentType;
if (string.IsNullOrEmpty(contentType.CharSet))
{
contentType.CharSet = "utf-8";
}
Copy the code– And the results are the same… Oh my God, does this site have an anti-crawler mechanism?? – So I analyzed the browser request and added the request header
httpClient.DefaultRequestHeaders.Add("Accept"."text/html,application/xhtml+xml,application/xml; Q = 0.9, image/webp, * / *; Q = 0.8");
httpClient.DefaultRequestHeaders.Referrer = new Uri("http://m.gxwztv.com/");
httpClient.DefaultRequestHeaders.Add("Cookie"."cids_AC=92542; cids_AC3=12743; cids_NU=12533; ras=41901%2C2121%2C12743%2C92542; cids_AC1=2121%2C41901; cv=5; fs=16; which=2");
httpClient.DefaultRequestHeaders.Add("User-Agent"."Mozilla / 5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1");
httpClient.DefaultRequestHeaders.Add("Accept-Language"."zh-CN,zh; Q = 0.8, en. Q = 0.6");
httpClient.DefaultRequestHeaders.Add("Connection"."keep-alive");
httpClient.DefaultRequestHeaders.Add("DNT"."1");
httpClient.DefaultRequestHeaders.Add("Accept-Encoding"."gzip, deflate, sdch");
Copy the codeFinally, I got it right. Then F5 runs… Oh, my God, it’s still wrong…
– Then I fumbled about for another ten minutes or so and finally glanced at the top line of code… Right! That’s right, that Gzip… I wonder if it’s to decompress… I’m a Web programmer. Usually the browser takes care of it for me. I don’t know. – Then open the almighty MSDN… Httclient unzip the Gzip API…
var handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip
};
HttpClient httpClient = new HttpClient(handler);
Copy the code