Mercurial > personal > weatherlog

annotate weatherlog/http_writer.py @ 18:9daa281d996b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add a HTTP timeout, and verify the server response.
author Paul Fisher <paul@pfish.zone>
date Thu, 16 Jan 2020 23:52:30 -0500
parents c01f9929ae38
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
1 """RemoteWriter which writes BSON to the given HTTP destination."""
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
2
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
3 import typing as t
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
4
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
5 import requests
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
6
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
7 from . import logger
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
8
18
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
9 TIMEOUT_SECS = 60
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
10
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
11
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
12 class HTTPWriter(logger.RemoteWriter):
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
13
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
14 def __init__(self, url: str, preamble: t.Dict[str, object]):
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
15 self._url = url
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
16 self._session = requests.Session()
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
17 self._session.headers['User-Agent'] = 'weatherlogger/0.0.1'
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
18 self._preamble = logger.bson_encode(preamble)
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
19
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
20 def write(self, readings: t.Iterable[t.Dict[str, object]]) -> None:
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
21 try:
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
22 data = b''.join(map(logger.bson_encode, readings))
18
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
23 response = self._session.post(
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
24 self._url, data=self._preamble + data, timeout=TIMEOUT_SECS)
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
25 response.raise_for_status()
18
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
26 contents = response.json()
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
27 if contents.get('status') != 'OK':
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
28 raise logger.RemoteWriteError(
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
29 f'Invalid server response: {contents!r}')
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
30 except requests.exceptions.RequestException as rex:
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
31 raise logger.RemoteWriteError("Coulndn't write values") from rex
18
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
32 except ValueError as ve:
9daa281d996b Add a HTTP timeout, and verify the server response.
Paul Fisher <paul@pfish.zone>
parents: 14
diff changeset
33 raise logger.RemoteWriteError('Invalid JSON response') from ve