cloudevents version 1.0.1 release (#102)

* docs: rename receiving cloudevents (#91)

Signed-off-by: Grant Timmerman <timmerman+devrel@google.com>

* add coc ref (#90)

Signed-off-by: Doug Davis <dug@us.ibm.com>

Co-authored-by: Curtis Mason <31265687+cumason123@users.noreply.github.com>

* CloudEvents equality override (#98)

* added tests to cloudevent eq

Signed-off-by: Curtis Mason <cumason@google.com>

* lint fix

Signed-off-by: Curtis Mason <cumason@google.com>

* modified changelog

Signed-off-by: Curtis Mason <cumason@google.com>

* version bump

Signed-off-by: Curtis Mason <cumason@google.com>

* cloudevent fields type checking adjustments (#97)

* added exceptions and more indepth can_read

Signed-off-by: Curtis Mason <cumason@google.com>

* moved is_binary, is_structured into http module

Signed-off-by: Curtis Mason <cumason@google.com>

* changelog and version bump

Signed-off-by: Curtis Mason <cumason@google.com>

* removed unused import and spacing

Signed-off-by: Curtis Mason <cumason@google.com>

* lint fix

Signed-off-by: Curtis Mason <cumason@google.com>

* reverted auto format change

Signed-off-by: Curtis Mason <cumason@google.com>

* reverted changelog and auto format changes

Signed-off-by: Curtis Mason <cumason@google.com>

* changelog 1.0.1 update (#101)

Signed-off-by: Curtis Mason <cumason@google.com>

Co-authored-by: Grant Timmerman <timmerman@google.com>
Co-authored-by: Doug Davis <dug@us.ibm.com>
This commit is contained in:
Curtis Mason 2020-08-13 18:31:48 -04:00 committed by GitHub
parent 390134c2b9
commit d95b1303a9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 218 additions and 82 deletions

View File

@ -4,10 +4,21 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.0.1]
### Added
- CloudEvent exceptions and event type checking in http module ([#96])
- CloudEvent equality override ([#98])
## [1.0.0]
### Added
- Update types and handle data_base64 structured ([#34])
- Added a user friendly CloudEvent class with data validation ([#36])
- CloudEvent structured cloudevent support ([#47])
- Separated http methods into cloudevents.http module ([#60])
- Implemented to_json and from_json in http module ([#72])
### Fixed
- Fixed top level extensions bug ([#71])
### Removed
- Removed support for Cloudevents V0.2 and V0.1 ([#43])
@ -74,6 +85,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#23]: https://github.com/cloudevents/sdk-python/pull/23
[#25]: https://github.com/cloudevents/sdk-python/pull/25
[#27]: https://github.com/cloudevents/sdk-python/pull/27
[#34]: https://github.com/cloudevents/sdk-python/pull/34
[#36]: https://github.com/cloudevents/sdk-python/pull/36
[#43]: https://github.com/cloudevents/sdk-python/pull/43
[#47]: https://github.com/cloudevents/sdk-python/pull/47
[#60]: https://github.com/cloudevents/sdk-python/pull/60
[#71]: https://github.com/cloudevents/sdk-python/pull/71
[#72]: https://github.com/cloudevents/sdk-python/pull/72
[#96]: https://github.com/cloudevents/sdk-python/pull/96
[#98]: https://github.com/cloudevents/sdk-python/pull/98

View File

@ -64,7 +64,7 @@ requests.post("<some-url>", data=body, headers=headers)
You can find a complete example of turning a CloudEvent into a HTTP request [in the samples directory](samples/http-json-cloudevents/client.py).
#### Request to CloudEvent
## Receiving CloudEvents
The code below shows how to consume a cloudevent using the popular python web framework
[flask](https://flask.palletsprojects.com/en/1.1.x/quickstart/):
@ -120,6 +120,17 @@ the same API. It will use semantic versioning with following rules:
- Email: https://lists.cncf.io/g/cncf-cloudevents-sdk
- Contact for additional information: Denis Makogon (`@denysmakogon` on slack).
Each SDK may have its own unique processes, tooling and guidelines, common
governance related material can be found in the
[CloudEvents `community`](https://github.com/cloudevents/spec/tree/master/community)
directory. In particular, in there you will find information concerning
how SDK projects are
[managed](https://github.com/cloudevents/spec/blob/master/community/SDK-GOVERNANCE.md),
[guidelines](https://github.com/cloudevents/spec/blob/master/community/SDK-maintainer-guidelines.md)
for how PR reviews and approval, and our
[Code of Conduct](https://github.com/cloudevents/spec/blob/master/community/GOVERNANCE.md#additional-information)
information.
## Maintenance
We use black and isort for autoformatting. We setup a tox environment to reformat

View File

@ -1 +1 @@
__version__ = "1.0.0"
__version__ = "1.0.1"

19
cloudevents/exceptions.py Normal file
View File

@ -0,0 +1,19 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
class CloudEventMissingRequiredFields(Exception):
pass
class CloudEventTypeErrorRequiredFields(Exception):
pass

View File

@ -15,6 +15,7 @@ import json
import typing
from cloudevents.http.event import CloudEvent
from cloudevents.http.event_type import is_binary, is_structured
from cloudevents.http.http_methods import (
from_http,
to_binary_http,

View File

@ -16,6 +16,7 @@ import datetime
import typing
import uuid
import cloudevents.exceptions as cloud_exceptions
from cloudevents.http.mappings import _required_by_version
@ -57,17 +58,20 @@ class CloudEvent:
).isoformat()
if self._attributes["specversion"] not in _required_by_version:
raise ValueError(
raise cloud_exceptions.CloudEventMissingRequiredFields(
f"Invalid specversion: {self._attributes['specversion']}"
)
# There is no good way to default 'source' and 'type', so this
# checks for those (or any new required attributes).
required_set = _required_by_version[self._attributes["specversion"]]
if not required_set <= self._attributes.keys():
raise ValueError(
raise cloud_exceptions.CloudEventMissingRequiredFields(
f"Missing required keys: {required_set - attributes.keys()}"
)
def __eq__(self, other):
return self.data == other.data and self._attributes == other._attributes
# Data access is handled via `.data` member
# Attribute access is managed via Mapping type
def __getitem__(self, key):

View File

@ -0,0 +1,29 @@
import typing
from cloudevents.sdk.converters import binary, structured
def is_binary(headers: typing.Dict[str, str]) -> bool:
"""Uses internal marshallers to determine whether this event is binary
:param headers: the HTTP headers
:type headers: typing.Dict[str, str]
:returns bool: returns a bool indicating whether the headers indicate a binary event type
"""
headers = {key.lower(): value for key, value in headers.items()}
content_type = headers.get("content-type", "")
binary_parser = binary.BinaryHTTPCloudEventConverter()
return binary_parser.can_read(content_type=content_type, headers=headers)
def is_structured(headers: typing.Dict[str, str]) -> bool:
"""Uses internal marshallers to determine whether this event is structured
:param headers: the HTTP headers
:type headers: typing.Dict[str, str]
:returns bool: returns a bool indicating whether the headers indicate a structured event type
"""
headers = {key.lower(): value for key, value in headers.items()}
content_type = headers.get("content-type", "")
structured_parser = structured.JSONHTTPCloudEventConverter()
return structured_parser.can_read(
content_type=content_type, headers=headers
)

View File

@ -1,7 +1,9 @@
import json
import typing
import cloudevents.exceptions as cloud_exceptions
from cloudevents.http.event import CloudEvent
from cloudevents.http.event_type import is_binary, is_structured
from cloudevents.http.mappings import _marshaller_by_format, _obj_by_version
from cloudevents.http.util import _json_or_string
from cloudevents.sdk import converters, marshaller, types
@ -27,19 +29,23 @@ def from_http(
marshall = marshaller.NewDefaultHTTPMarshaller()
if converters.is_binary(headers):
if is_binary(headers):
specversion = headers.get("ce-specversion", None)
else:
raw_ce = json.loads(data)
specversion = raw_ce.get("specversion", None)
if specversion is None:
raise ValueError("could not find specversion in HTTP request")
raise cloud_exceptions.CloudEventMissingRequiredFields(
"could not find specversion in HTTP request"
)
event_handler = _obj_by_version.get(specversion, None)
if event_handler is None:
raise ValueError(f"found invalid specversion {specversion}")
raise cloud_exceptions.CloudEventTypeErrorRequiredFields(
f"found invalid specversion {specversion}"
)
event = marshall.FromRequest(
event_handler(), headers, data, data_unmarshaller=data_unmarshaller
@ -71,7 +77,7 @@ def _to_http(
data_marshaller = _marshaller_by_format[format]
if event._attributes["specversion"] not in _obj_by_version:
raise ValueError(
raise cloud_exceptions.CloudEventTypeErrorRequiredFields(
f"Unsupported specversion: {event._attributes['specversion']}"
)

View File

@ -11,36 +11,7 @@
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import typing
from cloudevents.sdk.converters import binary, structured
TypeBinary = binary.BinaryHTTPCloudEventConverter.TYPE
TypeStructured = structured.JSONHTTPCloudEventConverter.TYPE
def is_binary(headers: typing.Dict[str, str]) -> bool:
"""Uses internal marshallers to determine whether this event is binary
:param headers: the HTTP headers
:type headers: typing.Dict[str, str]
:returns bool: returns a bool indicating whether the headers indicate a binary event type
"""
headers = {key.lower(): value for key, value in headers.items()}
content_type = headers.get("content-type", "")
binary_parser = binary.BinaryHTTPCloudEventConverter()
return binary_parser.can_read(content_type=content_type, headers=headers)
def is_structured(headers: typing.Dict[str, str]) -> bool:
"""Uses internal marshallers to determine whether this event is structured
:param headers: the HTTP headers
:type headers: typing.Dict[str, str]
:returns bool: returns a bool indicating whether the headers indicate a structured event type
"""
headers = {key.lower(): value for key, value in headers.items()}
content_type = headers.get("content-type", "")
structured_parser = structured.JSONHTTPCloudEventConverter()
return structured_parser.can_read(
content_type=content_type, headers=headers
)

View File

@ -16,7 +16,7 @@ import typing
from cloudevents.sdk import exceptions, types
from cloudevents.sdk.converters import base
from cloudevents.sdk.converters.structured import JSONHTTPCloudEventConverter
from cloudevents.sdk.converters.util import has_binary_headers
from cloudevents.sdk.event import base as event_base
from cloudevents.sdk.event import v1, v03
@ -28,13 +28,11 @@ class BinaryHTTPCloudEventConverter(base.Converter):
def can_read(
self,
content_type: str,
content_type: str = None,
headers: typing.Dict[str, str] = {"ce-specversion": None},
) -> bool:
return ("ce-specversion" in headers) and not (
isinstance(content_type, str)
and content_type.startswith(JSONHTTPCloudEventConverter.MIME_TYPE)
)
return has_binary_headers(headers)
def event_supported(self, event: object) -> bool:
return type(event) in self.SUPPORTED_VERSIONS

View File

@ -16,23 +16,24 @@ import typing
from cloudevents.sdk import types
from cloudevents.sdk.converters import base
from cloudevents.sdk.converters.util import has_binary_headers
from cloudevents.sdk.event import base as event_base
# TODO: Singleton?
class JSONHTTPCloudEventConverter(base.Converter):
TYPE = "structured"
MIME_TYPE = "application/cloudevents+json"
def can_read(
self,
content_type: str,
headers: typing.Dict[str, str] = {"ce-specversion": None},
self, content_type: str, headers: typing.Dict[str, str] = {},
) -> bool:
return (
isinstance(content_type, str)
and content_type.startswith(self.MIME_TYPE)
) or ("ce-specversion" not in headers)
or not has_binary_headers(headers)
)
def event_supported(self, event: object) -> bool:
# structured format supported by both spec 0.1 and 0.2

View File

@ -0,0 +1,10 @@
import typing
def has_binary_headers(headers: typing.Dict[str, str]) -> bool:
return (
"ce-specversion" in headers
and "ce-source" in headers
and "ce-type" in headers
and "ce-id" in headers
)

View File

@ -16,10 +16,12 @@ import base64
import json
import typing
import cloudevents.exceptions as cloud_exceptions
from cloudevents.sdk import types
# TODO(slinkydeveloper) is this really needed?
class EventGetterSetter(object):
# ce-specversion
@ -159,6 +161,9 @@ class EventGetterSetter(object):
class BaseEvent(EventGetterSetter):
_ce_required_fields = set()
_ce_optional_fields = set()
def Properties(self, with_nullable=False) -> dict:
props = dict()
for name, value in self.__dict__.items():
@ -215,7 +220,9 @@ class BaseEvent(EventGetterSetter):
missing_fields = self._ce_required_fields - raw_ce.keys()
if len(missing_fields) > 0:
raise ValueError(f"Missing required attributes: {missing_fields}")
raise cloud_exceptions.CloudEventMissingRequiredFields(
f"Missing required attributes: {missing_fields}"
)
for name, value in raw_ce.items():
if name == "data":
@ -233,8 +240,16 @@ class BaseEvent(EventGetterSetter):
body: typing.Union[bytes, str],
data_unmarshaller: types.UnmarshallerType,
):
if "ce-specversion" not in headers:
raise ValueError("Missing required attribute: 'specversion'")
required_binary_fields = {
f"ce-{field}" for field in self._ce_required_fields
}
missing_fields = required_binary_fields - headers.keys()
if len(missing_fields) > 0:
raise cloud_exceptions.CloudEventMissingRequiredFields(
f"Missing required attributes: {missing_fields}"
)
for header, value in headers.items():
header = header.lower()
if header == "content-type":
@ -242,9 +257,6 @@ class BaseEvent(EventGetterSetter):
elif header.startswith("ce-"):
self.Set(header[3:], value)
self.Set("data", data_unmarshaller(body))
missing_attrs = self._ce_required_fields - self.Properties().keys()
if len(missing_attrs) > 0:
raise ValueError(f"Missing required attributes: {missing_attrs}")
def MarshalBinary(
self, data_marshaller: types.MarshallerType

View File

@ -0,0 +1,71 @@
import pytest
from cloudevents.http import CloudEvent
@pytest.mark.parametrize("specversion", ["0.3", "1.0"])
def test_http_cloudevent_equality(specversion):
attributes = {
"source": "<source>",
"specversion": specversion,
"id": "my-id",
"time": "tomorrow",
"type": "tests.cloudevents.override",
"datacontenttype": "application/json",
"subject": "my-subject",
}
data = '{"name":"john"}'
event1 = CloudEvent(attributes, data)
event2 = CloudEvent(attributes, data)
assert event1 == event2
# Test different attributes
for key in attributes:
if key == "specversion":
continue
else:
attributes[key] = f"noise-{key}"
event3 = CloudEvent(attributes, data)
event2 = CloudEvent(attributes, data)
assert event2 == event3
assert event1 != event2 and event3 != event1
# Test different data
data = '{"name":"paul"}'
event3 = CloudEvent(attributes, data)
event2 = CloudEvent(attributes, data)
assert event2 == event3
assert event1 != event2 and event3 != event1
@pytest.mark.parametrize("specversion", ["0.3", "1.0"])
def test_http_cloudevent_mutates_equality(specversion):
attributes = {
"source": "<source>",
"specversion": specversion,
"id": "my-id",
"time": "tomorrow",
"type": "tests.cloudevents.override",
"datacontenttype": "application/json",
"subject": "my-subject",
}
data = '{"name":"john"}'
event1 = CloudEvent(attributes, data)
event2 = CloudEvent(attributes, data)
event3 = CloudEvent(attributes, data)
assert event1 == event2
# Test different attributes
for key in attributes:
if key == "specversion":
continue
else:
event2[key] = f"noise-{key}"
event3[key] = f"noise-{key}"
assert event2 == event3
assert event1 != event2 and event3 != event1
# Test different data
event2.data = '{"name":"paul"}'
event3.data = '{"name":"paul"}'
assert event2 == event3
assert event1 != event2 and event3 != event1

View File

@ -20,9 +20,11 @@ import json
import pytest
from sanic import Sanic, response
import cloudevents.exceptions as cloud_exceptions
from cloudevents.http import (
CloudEvent,
from_http,
is_binary,
to_binary_http,
to_structured_http,
)
@ -47,7 +49,7 @@ invalid_test_headers = [
},
]
invalid_cloudevent_request_bodie = [
invalid_cloudevent_request_body = [
{
"source": "<event-source>",
"type": "cloudevent.event.type",
@ -87,21 +89,22 @@ async def echo(request):
return response.raw(data, headers={k: event[k] for k in event})
@pytest.mark.parametrize("body", invalid_cloudevent_request_bodie)
@pytest.mark.parametrize("body", invalid_cloudevent_request_body)
def test_missing_required_fields_structured(body):
with pytest.raises((TypeError, NotImplementedError)):
with pytest.raises(cloud_exceptions.CloudEventMissingRequiredFields):
# CloudEvent constructor throws TypeError if missing required field
# and NotImplementedError because structured calls aren't
# implemented. In this instance one of the required keys should have
# prefix e-id instead of ce-id therefore it should throw
_ = from_http(
json.dumps(body), attributes={"Content-Type": "application/json"}
json.dumps(body),
headers={"Content-Type": "application/cloudevents+json"},
)
@pytest.mark.parametrize("headers", invalid_test_headers)
def test_missing_required_fields_binary(headers):
with pytest.raises((ValueError)):
with pytest.raises(cloud_exceptions.CloudEventMissingRequiredFields):
# CloudEvent constructor throws TypeError if missing required field
# and NotImplementedError because structured calls aren't
# implemented. In this instance one of the required keys should have
@ -165,7 +168,7 @@ def test_emit_structured_event(specversion):
@pytest.mark.parametrize("specversion", ["1.0", "0.3"])
def test_roundtrip_non_json_event(converter, specversion):
input_data = io.BytesIO()
for i in range(100):
for _ in range(100):
for j in range(20):
assert 1 == input_data.write(j.to_bytes(1, byteorder="big"))
compressed_data = bz2.compress(input_data.getvalue())
@ -201,7 +204,7 @@ def test_missing_ce_prefix_binary_event(specversion):
# breaking prefix e.g. e-id instead of ce-id
prefixed_headers[key[1:]] = headers[key]
with pytest.raises(ValueError):
with pytest.raises(cloud_exceptions.CloudEventMissingRequiredFields):
# CloudEvent constructor throws TypeError if missing required field
# and NotImplementedError because structured calls aren't
# implemented. In this instance one of the required keys should have
@ -278,7 +281,7 @@ def test_empty_data_structured_event(specversion):
# Testing if cloudevent breaks when no structured data field present
attributes = {
"specversion": specversion,
"datacontenttype": "application/json",
"datacontenttype": "application/cloudevents+json",
"type": "word.found.name",
"id": "96fb5f0b-001e-0108-6dfe-da6e2806f124",
"time": "2018-10-23T12:28:22.4579346Z",
@ -308,7 +311,6 @@ def test_empty_data_binary_event(specversion):
def test_valid_structured_events(specversion):
# Test creating multiple cloud events
events_queue = []
headers = {}
num_cloudevents = 30
for i in range(num_cloudevents):
event = {
@ -335,9 +337,6 @@ def test_valid_structured_events(specversion):
@pytest.mark.parametrize("specversion", ["1.0", "0.3"])
def test_structured_no_content_type(specversion):
# Test creating multiple cloud events
events_queue = []
headers = {}
num_cloudevents = 30
data = {
"id": "id",
"source": "source.com.test",
@ -362,28 +361,15 @@ def test_is_binary():
"ce-specversion": "1.0",
"Content-Type": "text/plain",
}
assert converters.is_binary(headers)
assert is_binary(headers)
headers = {
"Content-Type": "application/cloudevents+json",
}
assert not converters.is_binary(headers)
assert not is_binary(headers)
headers = {}
assert not converters.is_binary(headers)
def test_is_structured():
headers = {
"Content-Type": "application/cloudevents+json",
}
assert converters.is_structured(headers)
headers = {}
assert converters.is_structured(headers)
headers = {"ce-specversion": "1.0"}
assert not converters.is_structured(headers)
assert not is_binary(headers)
@pytest.mark.parametrize("specversion", ["1.0", "0.3"])