Mercurial > personal > weatherlog

annotate weatherlog/http_writer.py @ 14:c01f9929ae38

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Make logger and HTTP writer more general and resilient. This makes the logger and HTTP writer more general, by removing any dependency upon the exact data type they are writing. They can now handle any type of BSON-serializable dict, and track what they have sent by keeping track of the last *byte*, not the last timestamp.
author Paul Fisher <paul@pfish.zone>
date Tue, 15 Oct 2019 22:40:24 -0400
parents 6dbd9825b3f5
children 9daa281d996b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
1 """RemoteWriter which writes BSON to the given HTTP destination."""
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
2
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
3 import typing as t
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
4
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
5 import requests
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
6
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
7 from . import logger
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
8
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
9
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
10 class HTTPWriter(logger.RemoteWriter):
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
11
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
12 def __init__(self, url: str, preamble: t.Dict[str, object]):
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
13 self._url = url
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
14 self._session = requests.Session()
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
15 self._session.headers['User-Agent'] = 'weatherlogger/0.0.1'
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
16 self._preamble = logger.bson_encode(preamble)
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
17
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
18 def write(self, readings: t.Iterable[t.Dict[str, object]]) -> None:
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
19 try:
14
c01f9929ae38 Make logger and HTTP writer more general and resilient.
Paul Fisher <paul@pfish.zone>
parents: 11
diff changeset
20 data = b''.join(map(logger.bson_encode, readings))
11
6dbd9825b3f5 http_writer: switch to "preamble" based location/password.
Paul Fisher <paul@pfish.zone>
parents: 10
diff changeset
21 response = self._session.post(self._url, data=self._preamble + data)
10
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
22 response.raise_for_status()
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
23 except requests.exceptions.RequestException as rex:
b1657de734d5 Add HTTPWriter, which actually writes over HTTP.
Paul Fisher <paul@pfish.zone>
parents:
diff changeset
24 raise logger.RemoteWriteError("Coulndn't write values") from rex