Linux cURL - Revision history

Pio2pio: /* Use curl in shell scripts */

2021-05-22T11:03:33Z

Use curl in shell scripts

← Older revision		Revision as of 13:03, 22 May 2021
Line 45:		Line 45:
	== Use curl in shell scripts ==		== Use curl in shell scripts ==
	When using Curl in shell scripts, always pass <code>-fSSL</code>, which:		When using Curl in shell scripts, always pass <code>-fSSL</code>, which:
	* ~~Treats non-2xx/3xx responses as errors (~~<code>-f</code>).		* <code>-f</code> treats non-2xx/3xx responses as errors
	* ~~Disables the progress meter (~~<code>-sS</code>).		* <code>-sS</code> disables the progress meter
	* ~~Handles HTTP redirects (~~<code>-L</code>).		* <code>-L</code> handles HTTP redirects

	Occasionally, if you consider some <code>non-2xx/3xx</code> responses to be non-fatal, then omit <code>-f</code> then use logic to treat these responses as success.		Occasionally, if you consider some <code>non-2xx/3xx</code> responses to be non-fatal, then omit <code>-f</code> then use logic to treat these responses as success.

Pio2pio: /* Resolve DNS without using local cache */

2021-05-22T10:20:44Z

Resolve DNS without using local cache

← Older revision		Revision as of 12:20, 22 May 2021
Line 27:		Line 27:
	== Resolve DNS without using local cache ==		== Resolve DNS without using local cache ==
	<source lang=bash>		<source lang=bash>
	# ~~This~~ solution still uses local cache		sudo systemd-resolve --flush-caches # if you wish to flush dns
			DOMAIN=tvp.info; curl -I --resolve "$DOMAIN:80:$(dig +short $DOMAIN @8.8.8.8 \| head -1)" http://$DOMAIN
			systemd-resolve --statistics \| grep 'Current Cache Size' # should still be -> Current Cache Size: 0

			# --dns-servers - this solution still uses local cache
	curl --dns-servers _MY_DNS_SERVER_IP_ https://_MY_DOMAINS_		curl --dns-servers _MY_DNS_SERVER_IP_ https://_MY_DOMAINS_
	curl --dns-servers 8.8.8.8 https://tvp.pl		curl --dns-servers 8.8.8.8 https://tvp.info
	</source>		</source>
	{{Note\|libcurl and c-ares backend		{{Note\|libcurl and c-ares backend

Pio2pio: /* Always resolve DNS without local cache */

2021-05-22T09:35:28Z

Always resolve DNS without local cache

← Older revision		Revision as of 11:35, 22 May 2021
Line 25:		Line 25:
	</source>		</source>

	== ~~Always resolve~~ DNS without local cache ==		== Resolve DNS without using local cache ==
	<source lang=bash>		<source lang=bash>
			# This solution still uses local cache
	curl --dns-servers _MY_DNS_SERVER_IP_ https://_MY_DOMAINS_		curl --dns-servers _MY_DNS_SERVER_IP_ https://_MY_DOMAINS_
	curl --dns-servers 8.8.8.8 https://tvp.pl		curl --dns-servers 8.8.8.8 https://tvp.pl
Line 37:		Line 38:
	This option requires that libcurl was built with a resolver backend that supports this operation.		This option requires that libcurl was built with a resolver backend that supports this operation.
	The c-ares backend is the only such one. (Added in 7.33.0)</source>}}		The c-ares backend is the only such one. (Added in 7.33.0)</source>}}

	== Use curl in shell scripts ==		== Use curl in shell scripts ==
	When using Curl in shell scripts, always pass <code>-fSSL</code>, which:		When using Curl in shell scripts, always pass <code>-fSSL</code>, which:

Pio2pio: /* Use curl in shell scripts */

2021-05-22T09:06:38Z

Use curl in shell scripts

@@ Line 25: / Line 25: @@
 </source>
 == Use curl in shell scripts ==
 When using Curl in shell scripts, always pass <code>-fSSL</code>, which:

Pio2pio: /* Single website pinning */

2021-03-19T08:20:29Z

Single website pinning

← Older revision		Revision as of 10:20, 19 March 2021
Line 7:		Line 7:
	= Common curls =		= Common curls =
	== Single website pinning ==		== Single website pinning ==
			* [https://daniel.haxx.se/blog/2018/04/05/curl-another-host/ Fake the host header]
	Debugging problems with a site that's already live, it's helpful to bypass CDN and proxy layers when requesting content from the site by sending those web requests directly to a specific IP address without using the site's public DNS records. This practice of "pinning" a web request directly to a server can be accomplished by changing your <code>/etc/hosts file</code>, which will cause requests for a specified domain name (e.g: "www.example.com ") to be routed from your local machine to a specified IP address (e.g: <code>127.0.0.1</code>) until the changes you've made to <code>/etc/hosts</code> are reverted.		Debugging problems with a site that's already live, it's helpful to bypass CDN and proxy layers when requesting content from the site by sending those web requests directly to a specific IP address without using the site's public DNS records. This practice of "pinning" a web request directly to a server can be accomplished by changing your <code>/etc/hosts file</code>, which will cause requests for a specified domain name (e.g: "www.example.com ") to be routed from your local machine to a specified IP address (e.g: <code>127.0.0.1</code>) until the changes you've made to <code>/etc/hosts</code> are reverted.

Line 23:		Line 24:
	# --resolve [DOMAIN]:[PORT]:[IP] # the option format		# --resolve [DOMAIN]:[PORT]:[IP] # the option format
	</source>		</source>

	== Use curl in shell scripts ==		== Use curl in shell scripts ==
	When using Curl in shell scripts, always pass <code>-fSSL</code>, which:		When using Curl in shell scripts, always pass <code>-fSSL</code>, which:

Pio2pio: /* Use curl in shell scripts */

2020-10-18T15:46:13Z

Use curl in shell scripts

← Older revision		Revision as of 17:46, 18 October 2020
Line 41:		Line 41:

	Resources:		Resources:
	*[https://oncletom.io/2016/pipelining-http/ HTTP to HTTP with bash, curl and jq]		*[https://oncletom.io/2016/pipelining-http/ Processing HTTP to HTTP with bash, curl and jq]

	== Probe to monitor an endpoint ==		== Probe to monitor an endpoint ==

Pio2pio: /* Use curl in shell scripts */

2020-10-18T15:45:43Z

Use curl in shell scripts

← Older revision		Revision as of 17:45, 18 October 2020
Line 41:		Line 41:

	Resources:		Resources:
	*[https://oncletom.io/2016/pipelining-http/] HTTP to HTTP with bash, curl and jq		*[https://oncletom.io/2016/pipelining-http/ HTTP to HTTP with bash, curl and jq]

	== Probe to monitor an endpoint ==		== Probe to monitor an endpoint ==

Pio2pio: /* Use curl in shell scripts */

2020-10-18T15:45:28Z

Use curl in shell scripts

← Older revision		Revision as of 17:45, 18 October 2020
Line 38:		Line 38:
	# -o <filename> write the HTTP response body to the given file, not to standard output		# -o <filename> write the HTTP response body to the given file, not to standard output
	</source>		</source>


			Resources:
			*[https://oncletom.io/2016/pipelining-http/] HTTP to HTTP with bash, curl and jq

	== Probe to monitor an endpoint ==		== Probe to monitor an endpoint ==
	<source lang=bash>		<source lang=bash>

Pio2pio: /* Use curl in shell scripts */

2020-09-13T20:51:56Z

Use curl in shell scripts

← Older revision		Revision as of 22:51, 13 September 2020
Line 37:		Line 37:
	# -w '%{http_code}' write the HTTP response code to standard output		# -w '%{http_code}' write the HTTP response code to standard output
	# -o <filename> write the HTTP response body to the given file, not to standard output		# -o <filename> write the HTTP response body to the given file, not to standard output
			</source>
			== Probe to monitor an endpoint ==
			<source lang=bash>
			curl --connect-timeout 5 --retry 12 --retry-delay 1 --retry-max-time 60 service-1.acme.com:9000/sessions/new
	</source>		</source>

Pio2pio: /* Basics of HTTP */

2020-07-04T05:46:40Z

Basics of HTTP

← Older revision		Revision as of 07:46, 4 July 2020
Line 3:		Line 3:

	=[https://dev.to/inambe/how-http-works-2492 Basics of HTTP]=		=[https://dev.to/inambe/how-http-works-2492 Basics of HTTP]=
			* [https://www.haproxy.org/download/2.0/doc/configuration.txt haproxy configuration] Fantastic first section on how http works, header format and assembly when passing proxy servers

	= Common curls =		= Common curls =