Merge pull request #1 from msfussell/docstruc
First doc structure commit
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,11 @@
|
|||
# Debugging and Troubleshooting
|
||||
|
||||
This section describes different tools, techniques and common problems to help users debug and diagnose issues with Actions.
|
||||
|
||||
1. [Logs](logs.md)
|
||||
2. [Tracing and diagnostics](tracing.md)
|
||||
3. [Profiling and debugging](profiling_debugging.md)
|
||||
4. [Common issues](common_issues.md)
|
||||
|
||||
Please open a new Bug or Feature Request item on our [issues section](https://github.com/actionscore/actions/issues) if you've encountered a problem running Actions.
|
||||
If a security vulnerability has been found, contact the [Actions team](actionsct@microsoft.com).
|
|
@ -0,0 +1,127 @@
|
|||
# Common Issues
|
||||
|
||||
This section will walk you through some common issues and problems.
|
||||
|
||||
### I don't see the Actions sidecar injected to my pod
|
||||
|
||||
There could be several reasons to why a sidecar will not be injected into a pod.
|
||||
First, check your Deployment or Pod YAML file, and check that you have the following annotations in the right place:
|
||||
|
||||
Sample deployment:
|
||||
|
||||
<pre>
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nodeapp
|
||||
labels:
|
||||
app: node
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: node
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: node
|
||||
annotations:
|
||||
<b>actions.io/enabled: "true"</b>
|
||||
<b>actions.io/id: "nodeapp"</b>
|
||||
<b>actions.io/port: "3000"</b>
|
||||
spec:
|
||||
containers:
|
||||
- name: node
|
||||
image: actionscore.azurecr.io/samples/nodeapp
|
||||
ports:
|
||||
- containerPort: 3000
|
||||
imagePullPolicy: Always
|
||||
</pre>
|
||||
|
||||
If your pod spec template is annotated correctly and you still don't see the sidecar injected, make sure Actions was deployed to the cluster before your deployment or pod were deployed.
|
||||
|
||||
If this is the case, restarting the pods will fix the issue.
|
||||
|
||||
In order to further diagnose any issue, check the logs of the Actions sidecar injector:
|
||||
|
||||
```
|
||||
kubectl logs -l app=actions-sidecar-injector -n actions-system
|
||||
```
|
||||
|
||||
*Note: If you installed Actions to a different namespace, replace actions-system above with the desired namespace*
|
||||
|
||||
### I am unable to save state or get state
|
||||
|
||||
Have you installed an Actions State store in your cluster?
|
||||
|
||||
To check, use kubectl get a list of components:
|
||||
|
||||
`kubectl get components`
|
||||
|
||||
If there isn't a state store component, it means you need to set one up.
|
||||
Visit [here](../components/redis.md) for more details.
|
||||
|
||||
If everything's set up correctly, make sure you got the credentials right.
|
||||
Search the Actions runtime logs and look for any state store errors:
|
||||
|
||||
`kubectl logs <name-of-pod> actionsrt`.
|
||||
|
||||
### I am unable to publish and receive events
|
||||
|
||||
Have you installed an Actions Message Bus in your cluster?
|
||||
|
||||
To check, use kubectl get a list of components:
|
||||
|
||||
`kubectl get components`
|
||||
|
||||
If there isn't a pub-sub component, it means you need to set one up.
|
||||
Visit [here](../components/redis.md#configuring-redis-for-pubsub) for more details.
|
||||
|
||||
If everything's set up correctly, make sure you got the credentials right.
|
||||
Search the Actions runtime logs and look for any pub-sub errors:
|
||||
|
||||
`kubectl logs <name-of-pod> actionsrt`.
|
||||
|
||||
### The Actions Operator pod keeps crashing
|
||||
|
||||
Check that there's only one installation of the Actions Operator in your cluster.
|
||||
Find out by running `kubectl get pods -l app=actions-operator --all-namespaces`.
|
||||
|
||||
If two pods appear, delete the redundant Actions installation.
|
||||
|
||||
### I'm getting 500 Error responses when calling Actions
|
||||
|
||||
This means there are some internal issue inside the Actions runtime.
|
||||
To diagnose, view the logs of the sidecar:
|
||||
|
||||
`kubectl logs <name-of-pod> actionsrt`.
|
||||
|
||||
### I'm getting 404 Not Found responses when calling Actions
|
||||
|
||||
This means you're trying to call an Actions API endpoint that either doesn't exist or the URL is malformed.
|
||||
Look at the Actions API spec [here](https://github.com/actionscore/spec) and make sure you're calling the right endpoint.
|
||||
|
||||
### I don't see any incoming events or calls from other services
|
||||
|
||||
Have you specified the port your app is listening on?
|
||||
In Kubernetes, make sure the `actions.io/port` annotation is specified:
|
||||
|
||||
<pre>
|
||||
annotations:
|
||||
actions.io/enabled: "true"
|
||||
actions.io/id: "nodeapp"
|
||||
<b>actions.io/port: "3000"</b>
|
||||
</pre>
|
||||
|
||||
If using Actions Standalone and the Actions CLI, make sure you pass the `--app-port` flag to the `actions run` command.
|
||||
|
||||
### My Actions-enabled app isn't behaving correctly
|
||||
|
||||
The first thing to do is inspect the HTTP error code returned from the Actions API, if any.
|
||||
If you still can't find the issue, try enabling `debug` log levels for the Actions runtime. See [here](logs.md) how to do so.
|
||||
|
||||
You might also want to look at error logs from your own process. If running on Kubernetes, find the pod containing your app, and execute the following:
|
||||
|
||||
`kubectl logs <pod-name> <name-of-your-container>`.
|
||||
|
||||
If running in Standalone mode, you should see the stderr and stdout outputs from your app displayed in the main console session.
|
|
@ -0,0 +1,170 @@
|
|||
# Logs
|
||||
|
||||
This section will assist you in understanding how logging works in Actions, configuring and viewing logs.
|
||||
|
||||
## Overview
|
||||
|
||||
Logs have different, configurable verbosity levels.
|
||||
The levels outlined below are the same for both system components and the Actions sidecar process/container:
|
||||
|
||||
1. error
|
||||
2. warning
|
||||
3. info
|
||||
3. debug
|
||||
|
||||
error produces the minimum amount of output, where debug produces the maximum amount. The default level is info, which provides a balanced amount of information for operating Actions in normal conditions.
|
||||
|
||||
To set the output level, you can use the `--log-level` command-line option. For example:
|
||||
|
||||
`./actionsrt --log-level error` <br>
|
||||
`./placement --log-level debug`
|
||||
|
||||
This will start the Actions runtime binary with a log level of `error` and the Actions Actor Placement Service with a log level of `debug`.
|
||||
|
||||
## Configuring logs on Standalone Mode
|
||||
|
||||
As outlined above, every Actions binary takes a `--log-level` argument. For example, to launch the placement service with a log level of warning:
|
||||
|
||||
`./placement --log-level warning`
|
||||
|
||||
To set the log level when running your app with the Actions CLI, pass the `log-level` param:
|
||||
|
||||
`actions run --log-level warning node myapp.js`
|
||||
|
||||
## Viewing Logs on Standalone Mode
|
||||
|
||||
When running Actions with the Actions CLI, both your app's log output and the runtime's output will be redirected to the same session, for easy debugging.
|
||||
For example, this is the output when running Actions:
|
||||
|
||||
```
|
||||
actions run node myapp.js
|
||||
ℹ️ Starting Actions with id Trackgreat-Lancer on port 56730
|
||||
✅ You're up and running! Both Actions and your app logs will appear here.
|
||||
|
||||
== APP == App listening on port 3000!
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="starting Actions Runtime -- version 0.3.0-alpha -- commit b6f2810-dirty"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="log level set to: info"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="standalone mode configured"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="action id: Trackgreat-Lancer"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="loaded component statestore (state.redis)"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="loaded component messagebus (pubsub.redis)"
|
||||
== ACTIONS == 2019/09/05 12:26:43 redis: connecting to localhost:6379
|
||||
== ACTIONS == 2019/09/05 12:26:43 redis: connected to localhost:6379 (localAddr: [::1]:56734, remAddr: [::1]:6379)
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=warning msg="failed to init input bindings: app channel not initialized"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="actor runtime started. actor idle timeout: 1h0m0s. actor scan interval: 30s"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="actors: starting connection attempt to placement service at localhost:50005"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="http server is running on port 56730"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="gRPC server is running on port 56731"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="actions initialized. Status: Running. Init Elapsed 8.772922000000001ms"
|
||||
== ACTIONS == time="2019-09-05T12:26:43-07:00" level=info msg="actors: established connection to placement service at localhost:50005"
|
||||
```
|
||||
|
||||
## Configuring Logs on Kubernetes
|
||||
|
||||
This section shows you how to configure the log levels for Actions system pods and the Actions sidecar running on Kubernetes.
|
||||
|
||||
### Setting the sidecar log level
|
||||
|
||||
You can set the log level individually for every sidecar by providing the following annotation in your pod spec template:
|
||||
|
||||
```
|
||||
annotations:
|
||||
actions.io/log-level: "debug"
|
||||
```
|
||||
|
||||
### Setting system pods log level
|
||||
|
||||
When deploying Actions to your cluster using Helm, you can individually set the log level for every Actions system component.
|
||||
|
||||
#### Setting the Operator log level
|
||||
|
||||
`helm install actionscore/actions-operator -n actions --namespace actions-system --set actions_operator.logLevel=error`
|
||||
|
||||
#### Setting the Placement Service log level
|
||||
|
||||
`helm install actionscore/actions-placement -n actions --namespace actions-system --set actions_placement.logLevel=error`
|
||||
|
||||
#### Setting the Sidecar Injector log level
|
||||
|
||||
`helm install actionscore/actions-sidecar-injector -n actions --namespace actions-system --set actions_sidecar_injector.logLevel=error`
|
||||
|
||||
|
||||
|
||||
## Viewing Logs on Kubernetes
|
||||
|
||||
Actions logs are written to stdout and stderr.
|
||||
This section will guide you on how to view logs for Actions system components as well as the Actions sidecar.
|
||||
|
||||
### Sidecar Logs
|
||||
|
||||
When deployed in Kubernetes, the Actions sidecar injector will inject an Actions container named `actionsrt` into your annotated pod.
|
||||
In order to view logs for the sidecar, simply find the pod in question by running `kubectl get pods`:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
addapp-74b57fb78c-67zm6 2/2 Running 0 40h
|
||||
```
|
||||
|
||||
Next, get the logs for the Actions sidecar container:
|
||||
|
||||
`kubectl logs addapp-74b57fb78c-67zm6 -c actionsrt`
|
||||
```
|
||||
time="2019-09-04T02:52:27Z" level=info msg="starting Actions Runtime -- version 0.3.0-alpha -- commit b6f2810-dirty"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="log level set to: info"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="kubernetes mode configured"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="action id: addapp"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="application protocol: http. waiting on port 6000"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="application discovered on port 6000"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="actor runtime started. actor idle timeout: 1h0m0s. actor scan interval: 30s"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="actors: starting connection attempt to placement service at actions-placement.actions-system.svc.cluster.local:80"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="http server is running on port 3500"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="gRPC server is running on port 50001"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="actions initialized. Status: Running. Init Elapsed 64.234049ms"
|
||||
time="2019-09-04T02:52:27Z" level=info msg="actors: established connection to placement service at actions-placement.actions-system.svc.cluster.local:80"
|
||||
```
|
||||
|
||||
### System Logs
|
||||
|
||||
Actions runs the following system pods:
|
||||
|
||||
* Actions operator
|
||||
* Actions sidecar injector
|
||||
* Actions placement service
|
||||
|
||||
#### Viewing Operator Logs
|
||||
|
||||
```
|
||||
kubectl logs -l app=actions-operator -n actions-system
|
||||
time="2019-09-05T19:03:43Z" level=info msg="log level set to: info"
|
||||
time="2019-09-05T19:03:43Z" level=info msg="starting Actions Operator -- version 0.3.0-alpha -- commit b6f2810-dirty"
|
||||
time="2019-09-05T19:03:43Z" level=info msg="Actions Operator is started"
|
||||
```
|
||||
|
||||
*Note: If Actions is installed to a different namespace than actions-system, simply replace the namespace to the desired one in the command above*
|
||||
|
||||
#### Viewing Sidecar Injector Logs
|
||||
|
||||
```
|
||||
kubectl logs -l app=actions-sidecar-injector -n actions-system
|
||||
time="2019-09-03T21:01:12Z" level=info msg="log level set to: info"
|
||||
time="2019-09-03T21:01:12Z" level=info msg="starting Actions Sidecar Injector -- version 0.3.0-alpha -- commit b6f2810-dirty"
|
||||
time="2019-09-03T21:01:12Z" level=info msg="Sidecar injector is listening on :4000, patching Actions-enabled pods"
|
||||
```
|
||||
|
||||
*Note: If Actions is installed to a different namespace than actions-system, simply replace the namespace to the desired one in the command above*
|
||||
|
||||
#### Viewing Placement Service Logs
|
||||
|
||||
```
|
||||
kubectl logs -l app=actions-placement -n actions-system
|
||||
time="2019-09-03T21:01:12Z" level=info msg="log level set to: info"
|
||||
time="2019-09-03T21:01:12Z" level=info msg="starting Actions Placement Service -- version 0.3.0-alpha -- commit b6f2810-dirty"
|
||||
time="2019-09-03T21:01:12Z" level=info msg="placement Service started on port 50005"
|
||||
time="2019-09-04T00:21:57Z" level=info msg="host added: 10.244.1.89"
|
||||
```
|
||||
*Note: If Actions is installed to a different namespace than actions-system, simply replace the namespace to the desired one in the command above*
|
||||
|
||||
### Non Kubernetes Environments
|
||||
|
||||
The examples above are specific specific to Kubernetes, but the principal is the same for any kind of container based environment: simply grab the container ID of the Actions sidecar and/or system component (if applicable) and view its logs.
|
||||
|
|
@ -0,0 +1,107 @@
|
|||
# Profiling and Debugging
|
||||
|
||||
In any real world scenario, an app might start exhibiting undesirable behavior in terms of resource spikes.
|
||||
CPU/Memory spikes are not uncommon in most cases.
|
||||
|
||||
Actions allows users to start an on-demand profiling session using `pprof` through its profiling server endpoint and start an instrumentation session to discover problems and issues such as concurrency, performance, cpu and memory usage.
|
||||
|
||||
## Enable profiling
|
||||
|
||||
Actions allows you to enable profiling in both Kubernetes and Standalone modes.
|
||||
|
||||
### Kubernetes
|
||||
|
||||
To enable profiling in Kubernetes, simply add the following annotation to your Actions annotated pod:
|
||||
|
||||
<pre>
|
||||
annotations:
|
||||
actions.io/enabled: "true"
|
||||
actions.io/id: "rust-app"
|
||||
<b>actions.io/profiling: "true"</b>
|
||||
</pre>
|
||||
|
||||
### Standalone
|
||||
|
||||
To enable profiling in Standalone mode, pass the `enable-profiling` and the `profile-port` flags to the Actions CLI:
|
||||
Note that `profile-port` is not required, and Actions will pick an available port.
|
||||
|
||||
`actions run --enable-profiling true --profile-port 7777 python myapp.py`
|
||||
|
||||
## Debug a profiling session
|
||||
|
||||
After profiloing is enabled, we can start a profiling session to investigate what's going on with the Actions runtime.
|
||||
|
||||
### Kubernetes
|
||||
|
||||
First, find the pod containing the Actions runtime. If you don't already know the the pod name, type `kubectl get pods`:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
divideapp-6dddf7dc74-6sq4l 2/2 Running 0 2d23h
|
||||
```
|
||||
|
||||
If profiling has been enabled successfully, the runtime logs should show the following:
|
||||
`time="2019-09-09T20:56:21Z" level=info msg="starting profiling server on port 7777"`
|
||||
|
||||
In this case, we want to start a session with the Actions runtime inside of pod `divideapp-6dddf7dc74-6sq4l`.
|
||||
|
||||
We can do so by connecting to the pod via port forwarding:
|
||||
|
||||
```
|
||||
kubectl port-forward divideapp-6dddf7dc74-6sq4 7777:7777
|
||||
Forwarding from 127.0.0.1:7777 -> 7777
|
||||
Forwarding from [::1]:7777 -> 7777
|
||||
Handling connection for 7777
|
||||
```
|
||||
|
||||
Now that the connection has been established, we can use `pprof` to profile the Actions runtime.
|
||||
|
||||
The following example will create a `cpu.pprof` file containing samples from a profile session that lasts 120 seconds:
|
||||
`curl "http://localhost:7777/debug/pprof/profile?seconds=120" > cpu.pprof`
|
||||
|
||||
Analyze the file with pprof:
|
||||
|
||||
```
|
||||
pprof cpu.pprof
|
||||
```
|
||||
|
||||
You can also save the results in a visualized way inside a PDF:
|
||||
|
||||
`go tool pprof --pdf your-binary-file http://localhost:7777/debug/pprof/profile?seconds=120 > profile.pdf`
|
||||
|
||||
For memory related issues, you can profile the heap:
|
||||
|
||||
`go tool pprof --pdf your-binary-file http://localhost:7777/debug/pprof/heap > heap.pdf`
|
||||
|
||||

|
||||
|
||||
Profiling allocated objects:
|
||||
|
||||
```
|
||||
go tool pprof http://localhost:7777/debug/pprof/heap
|
||||
> exit
|
||||
|
||||
Saved profile in /Users/myusername/pprof/pprof.actionsrt.alloc_objects.alloc_space.inuse_objects.inuse_space.003.pb.gz
|
||||
```
|
||||
|
||||
To analyze, grab the file path above (its a dynamic file path, so pay attention to note paste this one), and execute:
|
||||
|
||||
`go tool pprof -alloc_objects --pdf /Users/myusername/pprof/pprof.actionsrt.alloc_objects.alloc_space.inuse_objects.inuse_space.003.pb.gz > alloc-objects.pdf`
|
||||
|
||||

|
||||
|
||||
|
||||
### Standalone
|
||||
|
||||
For Standalone mode, locate the Actions instance that you want to profile:
|
||||
|
||||
```
|
||||
actions list
|
||||
APP ID ACTIONS PORT APP PORT COMMAND AGE CREATED PID
|
||||
node-subscriber 3500 3000 node app.js 12s 2019-09-09 15:11.24 896
|
||||
```
|
||||
|
||||
Grab the ACTIONS PORT, and if profiling has been enabled as desribed above, you can now start using `pprof` to profile Actions.
|
||||
Look at the Kubernetes examples above for some useful commands to profile Actions.
|
||||
|
||||
More info on pprof can be found [here](https://github.com/google/pprof).
|
|
@ -0,0 +1,124 @@
|
|||
# Tracing
|
||||
|
||||
Actions integrates seamlessly with Open Census for telemtry and tracing.
|
||||
Visit our [Distributed Tracing](../distributed_tracing.md) section for more info.
|
||||
|
||||
It is recommended to run Actions with tracing enabled for any production scenario.
|
||||
Since Actions uses Open Census, you can configure various exporters for tracing and telemtry data based on your environment, whether it is running in the cloud or on-premises.
|
||||
|
||||
## Distributed Tracing with Zipkin on Kubernetes
|
||||
|
||||
The following steps will show you how to configure Actions to send distributed tracing data to Zipkin running as a container in your Kubernetes cluster, and how to view them.
|
||||
|
||||
|
||||
### Setup
|
||||
|
||||
First, deploy Zipkin:
|
||||
|
||||
```
|
||||
kubectl run zipkin --image openzipkin/zipkin --port 9411
|
||||
```
|
||||
|
||||
Create a Kubernetes Service for the Zipkin pod:
|
||||
|
||||
```
|
||||
kubectl expose deploy zipkin --type ClusterIP --port 9411
|
||||
```
|
||||
|
||||
Next, create the following YAML file locally:
|
||||
|
||||
```
|
||||
apiVersion: actions.io/v1alpha1
|
||||
kind: Configuration
|
||||
metadata:
|
||||
name: zipkin
|
||||
spec:
|
||||
tracing:
|
||||
enabled: true
|
||||
exporterType: zipkin
|
||||
exporterAddress: "http://zipkin.default.svc.cluster.local:9411/api/v2/spans"
|
||||
expandParams: true
|
||||
includeBody: true
|
||||
```
|
||||
|
||||
Finally, deploy the Actions configuration:
|
||||
|
||||
```
|
||||
kubectl apply -f config.yaml
|
||||
```
|
||||
|
||||
In order to enable this configuration for your Actions sidecar, add the following annotation to your pod spec template:
|
||||
|
||||
```
|
||||
annotations:
|
||||
actions.io/config: "zipkin"
|
||||
```
|
||||
|
||||
That's it! your sidecar is now configured for use with Open Census and Zipkin.
|
||||
|
||||
### Viewing Tracing Data
|
||||
|
||||
To view traces, connect to the Zipkin Service and open the UI:
|
||||
|
||||
```
|
||||
kubectl port-forward svc/zipkin 9411:9411
|
||||
```
|
||||
|
||||
On your browser, go to ```http://localhost:9411``` and you should see the Zipkin UI.
|
||||
|
||||

|
||||
|
||||
## Distributed Tracing with Zipkin - Standalone Mode
|
||||
|
||||
For standalone mode, create an Actions Configuration CRD file locally and reference it with the Actions CLI.
|
||||
|
||||
1. Create the following YAML file:
|
||||
|
||||
```
|
||||
apiVersion: actions.io/v1alpha1
|
||||
kind: Configuration
|
||||
metadata:
|
||||
name: zipkin
|
||||
spec:
|
||||
tracing:
|
||||
enabled: true
|
||||
exporterType: zipkin
|
||||
exporterAddress: "http://localhost:9411/api/v2/spans"
|
||||
expandParams: true
|
||||
includeBody: true
|
||||
```
|
||||
|
||||
2. Launch Zipkin using Docker:
|
||||
|
||||
```
|
||||
docker run -d -p 9411:9411 openzipkin/zipkin
|
||||
```
|
||||
|
||||
3. Launch Actions with the `--config` param:
|
||||
|
||||
```
|
||||
actions run --app-id mynode --app-port 3000 --config ./cofig.yaml node app.js
|
||||
```
|
||||
|
||||
## Tracing Configuration
|
||||
|
||||
The `tracing` section under the `Configuration` spec contains the following properties:
|
||||
|
||||
```
|
||||
tracing:
|
||||
enabled: true
|
||||
exporterType: zipkin
|
||||
exporterAddress: ""
|
||||
expandParams: true
|
||||
includeBody: true
|
||||
```
|
||||
|
||||
The following table lists the different properties.
|
||||
|
||||
Property | Type | Description
|
||||
---- | ------- | -----------
|
||||
enabled | bool | Set tracing to be enabled or disabled
|
||||
exporterType | string | Name of the Open Census exporter to use. For example: Zipkin, Azure Monitor, etc
|
||||
exporterAddress | string | URL of the exporter
|
||||
expandParams | bool | When true, expands parameters passed to HTTP endpoints
|
||||
includeBody | bool | When true, includes the request body in the tracing event
|
|
@ -0,0 +1,65 @@
|
|||
# Actor Pattern with Actions
|
||||
|
||||
Actions is a new programming model that is based on RPC communication between the framework and the user code, namely HTTP. However, it does support developers to write their code using the Actor Pattern. Specifically, Actions allow a contextual Id to be associated with messages. Actions provides:
|
||||
|
||||
1) Routing by a contextual Id.
|
||||
2) Ensuring single-threaded access per contextual Id.
|
||||
3) Setting and getting state by a contextual Id is serialized.
|
||||
|
||||
The combination of above behaviors delivers essential characteristics of an Actor: uniquely identified, single-threaded, with isolated state.
|
||||
|
||||
## Authoring an Actor
|
||||
|
||||
In Actions, authoring Actor code is no difference with writing any other service code. An Actor in this case is a web server that listens to any number of messages the actor expects to handle.
|
||||
|
||||
When a request is routed to a handler of the web server, a contextual Id may present in the request header. Actions guarantees that requests with the same contextual Id are dispatched to the web server sequentially.
|
||||
|
||||
Hanlder code can set or retrieve state from the Actions sidecar with the contextual Id attached.
|
||||
|
||||
## Talking to an Actor
|
||||
|
||||
Just like invoking any other Actions instances, you can send a request to an Actor by doing a POST to the sidecar. The **to** address of your message will be in the format of:
|
||||
```yaml
|
||||
<Actions name>.<Contextual Id>
|
||||
```
|
||||
|
||||
> Actions uses **virtual actors** pattern. You don't need to explicitly create or destroy Actor instances.
|
||||
|
||||
For example, the following POST request to **/set-temperature** sends a temperature payload to a **theromstat** with id **123**:
|
||||
```json
|
||||
{
|
||||
"to": [
|
||||
"theromstat.123"
|
||||
],
|
||||
"data": {
|
||||
"temperature": 72
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Your code can access the response from the actor through the HTTP response body.
|
||||
|
||||
Actions also allows you to make direct calls to actors. The Actions sidecar provides a special **/actor** route that can be used to route requests to actor instance. For example, to invoke the above actor directly, send a POST request to:
|
||||
```bash
|
||||
http://localhost:<sidecarport>/actors/theromstat/123
|
||||
```
|
||||
|
||||
## Key Differences From a Full Actor Framework
|
||||
|
||||
Actions programming model is not an Actor programming model. Although it provides certain actor characteristics, Actions differ from common Actor programming model in several ways.
|
||||
|
||||
### Single Activation
|
||||
|
||||
Many Actor frameworks requires single activation of an actor at any time. Actions doesn’t offer such guarantees. There should be a single activation for the most of time. However, multiple activations may exist during failovers, scaling, and network partitions. Actions will converge back to single activation when such conditions are resolved.
|
||||
|
||||
Actions offers exact-once delivery within a configurable window. Actions delivers requests for the same actor id to the same service instance, while serializing client requests as well as state access of an instance. The combination of these features offers a high-fidelity simulation of the single activation behavior. However, there could be corner cases that cause problems when multiple activations do occur.
|
||||
|
||||
### State Transaction Scope
|
||||
|
||||
Some Actor frameworks allow wrapping multiple state operations into an atomic transaction. In Actions, each state operation is a separate transaction. Because Actions doesn't dictate how user code is written, a user may trigger multiple state transactions in her code. If the code crashes between transactions, the state is left at the last committed transaction.
|
||||
|
||||
|
||||
## More Information
|
||||
To learn more about Actions Actor Pattern support, consult the following topics:
|
||||
|
||||
* [Enabling the Actor Pattern](../../topics/enable_actor_pattern.md)
|
|
@ -0,0 +1,40 @@
|
|||
# Integration with Actor Frameworks
|
||||
This article introduces how Actions integrate with existing Actor Frameworks such as [Orleans](https://github.com/dotnet/orleans), [Akka](https://akka.io/), and [Service Fabric Reliable Actors](https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-actors-introduction). With Actions integration, these Actor frameworks will allow developers to write Actor code in any programming languages, and to plug their Actors into Actions eventing pipeline that enables reliable messaging, pub-sub, binding to event sources and many other features.
|
||||
|
||||
## Client-side integration
|
||||
On the cliend side, Actions acts as a shim in front of an Actor proxy. This allows Actions to bring reliable messaging, binding and other features to the connected Actor framework.
|
||||

|
||||
|
||||
## Server-side integration
|
||||
On the server side, Actions leverages Actor framework capabilities such as state management and single activation guarantee to offer complete Actor framework capabilities. Actions defines an **Actor API**, which the integrated Actor frameworks are supposed to implement/support.
|
||||
|
||||
> Actions comes with a built-in Actor framework implemenation through the same Actor API.
|
||||
|
||||
The following diagram illustrates two possible execution paths. 
|
||||
|
||||
### Client calling an Actor (green path)
|
||||
|
||||
1. A client makes a call to an Actor instance.
|
||||
2. Actions calls an Actor proxy to forward the request.
|
||||
3. As the Actor runtime activates/invokes an Actor, it calls Actions through the Actor API to invoke the user method.
|
||||
4. Actions forwards the request to user code through Actions API.
|
||||
5. As the response is returned, it was sent back to client as a response to the original request.
|
||||
|
||||
### A message triggering an Actor (yellow path)
|
||||
|
||||
1. A message is routed to the Actions instance that is hosting the destination Actor.
|
||||
2. Actions dispatches the message to the Actor runtime when possible.
|
||||
3. The rest of the steps are the same as the green path.
|
||||
|
||||
## Actor API
|
||||
At a high level, the Actor API shall contain the following methods:
|
||||
|
||||
* Dispatch to Actor when possible
|
||||
* Call method on an Actor now
|
||||
* Load/save state
|
||||
* Create/remove
|
||||
* Locate & send message to an Actor
|
||||
|
||||
## Custom Programming Experience
|
||||
Custom programming experiences can be built on top of Actions through a custom library. The Custom library adapts Actions API to a custom API shape that the user code consumes, as shown in the following diagram:
|
||||

|
|
@ -0,0 +1,77 @@
|
|||
# Enabling the Actor Pattern
|
||||
The Actor Pattern has been broadly used in modeling complex distributed systems in which a large number of individual agents work together. For example, in many IoT solutions, actors are often used as digital presences of physical devices. Actions supports the Actor Pattern by offering key characteristics of an actor. This document introduces architecturally how Actions enables the Actor Pattern. To learn about how to use the Actor Pattern with Actions, please read [this document](../concepts/actor/actor_pattern.md).
|
||||
|
||||
## Overview
|
||||
Actions uses an off-path **partition table** to assign actor ids to service instances so that requests to the same actor id is always routed to the same instance. The partition table is dynamically adjusted when re-partitioning or failover happens. Changes to the table are broadcasted to Actions sidecars to update their own settings to match with the global table.
|
||||
|
||||
A client can talk to an actor through either reliable messaging or direct invocation, as illustrated by the following diagram:
|
||||

|
||||
|
||||
### Calling an actor through messaging
|
||||
1. When a process pair of user code and Actions sidecar comes up, the sidecar registers the pair (such as a Pod on Kubernetes) with a **partition table** hosted on the Actions control plane.
|
||||
2. The partition table updates itself to reflect the topology change.
|
||||
Then, it broadcasts a change notification to all Actions sidecars.
|
||||
3. The Actions sidecar updates its own settings to poll messages from corresponding partitions of a reliable queue.
|
||||
4. The client code sends requests to the reliable queue, which puts the requests in corresponding partitions based on the associated contextual ids. Then, the messages are picked up and consumed during the next poll.
|
||||
|
||||
### Calling an actor through direct invocation
|
||||
1. The client sidecar reads the partition table to find actor id assignments.
|
||||
2. The sidecar figures out a direct route to the target service instance and sends the request direct to the destination sidecar.
|
||||
|
||||
## Messaging
|
||||
|
||||
When enabled, each Actions sidecar has a designated reliable message queue. This allows the sidecar to sequentialize requests to a specific actor id. Actions supports at-least-once delivery and exact-once delivery within a configurable time window. Please see [reliable messaging](TBD) for more details.
|
||||
|
||||
Reliable messing is not used for direct invocations. In such a case, user code is expected to handle response errors and retry the request as needed.
|
||||
|
||||
## State Management
|
||||
|
||||
All state access are encapsulated in a **state provider**. Actions expects a state provider to offer state partitioning for scale and replication for high availability and reliability.
|
||||
|
||||
Because all requests to an actor instance are routed to the same Actions service instance, the user code can use local variables as local caches of its state.
|
||||
|
||||
When Actions routes an actor instance to a new Actions service instance, it invokes the **/state** endpoint so that the actor instance can restore state from the state store.
|
||||
|
||||
## Partition Table
|
||||
|
||||
Actions uses different strategies to mange and use the partition table for different scenarios. Although partition table doesn’t sit on active data path, Actions tries to employ any possible optimizations to minimize the overheads of table management and lookups.
|
||||
|
||||
### Skipping the table
|
||||
|
||||
If an Actions service is stateless, and if only direct invocation is needed, the partition table is skipped. In this case, requests to actor instances are distributed by the service load balancer.
|
||||
|
||||
If an Actions service is stateless, and is known to have light traffic so that messaging system partition is not needed, the table can also be skipped even when reliable messaging is used, because all Actions instances will simply compete for the latest messages from the same partition.
|
||||
|
||||
If Actions services have stable identifiers (i.e. doesn’t change when restarted or relocated), and both the possible range of actor Ids and all Actions service ids are known, a predefined algorithm can be used to decide actor partitions instead of looking up the partition table. However, in such a case, dynamic partitioning is not allowed.
|
||||
|
||||
### Range-only table
|
||||
|
||||
Range-only table contains rows of actor id ranges only. When the service replica number changes, the table is updated so that the possible range is evenly distributed across all service replicas.
|
||||
|
||||
The partition table can change its assignments based on partition loads. For example, when it finds the first partition is heavily used than others, it can reassign multiple service instances to the partition through **dynamic assignments**, or re-partition through **dynamic ranges**, as shown by the following diagram:
|
||||
|
||||

|
||||
|
||||
Dynamic assignments requires less sidecar configuration changes, but it requires a sidecar being able to handle multiple partitions.
|
||||
|
||||
### Per-actor table
|
||||
|
||||
Per-actor table maintains a detailed mapping between actor ids and service instances. Per-actor table is quite expensive to maintain. It’s most useful when the actor code fluctuates in resource consumption when given different workloads. In such case, a more sophisticated scheduling algorithm, such as a greedy algorithm or a dynamic programming algorithm, can be used to place the actor instances.
|
||||
|
||||
In addition to the increased maintenance burden, per-actor table also triggers sidecar configuration updates much more frequently. And the sidecar needs to be able to handle scattered id ranges, when could be hard to be implemented against certain messaging systems.
|
||||
|
||||
In the above discussion, it's been assumed that service instances have stable identifiers. If this is not the case, a check-in-check-out mechanism is needed for a service instance to associate itself to a given partition. In such as case, the partition table defines partition ranges but doesn’t define assignments. Instead, all sidecars come in as competing consumers and attempt to check out partitions. A partition can be explicitly checked in for scenarios such controlled upgrades. Otherwise, a checkout expires after certain time window and the partition is made available for checkouts again.
|
||||
|
||||
## Sequential Access
|
||||
|
||||
When configured to run as actors, Actions sidecar dispatches messages to user code. The user code is free to use whatever threading model of choice to handle these messages. When the user code tries to preserve state, the state write requests are again dispatched to the underlying state store sequentially.
|
||||
|
||||
Actions allows read-only requests (signified by the GET verb) to be routed to any service instances. This allows the single-writer-multiple-reader pattern, in which readers get eventual consistent (but almost always up-to-date - the delta is the single in-flight transaction) reads on actor states.
|
||||
|
||||
Such sequential access pattern is broken when multiple activations occur. For example, due to network partition, two service instances get different partition table data and try to poll data from the same partition. One message for an actor goes to intance A and the subsquent message for the same actor goes to instance B. This may lead to consistency issues.
|
||||
|
||||
One way to partially mitigate this is to add a auto-increment version tag on state data so that the user code can check for conflicting updates while submitting its changes.
|
||||
|
||||
## Summary
|
||||
|
||||
Actions can be used as building blocks to build a complete Actor framework but itself isn’t one. As a developer, you should be aware that when you use Actions by itself, you can configure it to offer some characteristics of an Actor, but you should not assume it gives a complete feature set as a complete Actor framework.
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
After Width: | Height: | Size: 1.1 MiB |
After Width: | Height: | Size: 28 KiB |
After Width: | Height: | Size: 28 KiB |
After Width: | Height: | Size: 754 KiB |
After Width: | Height: | Size: 43 KiB |
After Width: | Height: | Size: 15 KiB |
After Width: | Height: | Size: 26 KiB |
After Width: | Height: | Size: 340 KiB |
After Width: | Height: | Size: 13 KiB |
After Width: | Height: | Size: 124 KiB |
After Width: | Height: | Size: 9.2 KiB |
After Width: | Height: | Size: 21 KiB |
After Width: | Height: | Size: 212 KiB |
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|
|
@ -0,0 +1,3 @@
|
|||
# documentation
|
||||
|
||||
Content for this file to be added
|