litecow package

Subpackages

• litecow.common package
  • Submodules
  • litecow.common.common module
  • litecow.common.litecow_pb2 module
  • litecow.common.litecow_pb2_grpc module
  • Module contents

Submodules

litecow.client module

class litecow.client.ICOWClient(channel)

Bases: object

get_inference(model_key: str, inputs: Union[Dict[str, numpy.ndarray], List[numpy.ndarray], numpy.ndarray], outputs: Optional[List[str]] = None, model_version: Optional[str] = None) → Dict[str, numpy.ndarray]

Get inference with input arrays

Parameters

• model_key (str) – The key to the model in s3.

• inputs (Union[Dict[str, np.ndarray], List[np.ndarray], np.ndarray]) – The input(s) to run inference on may be a dict mapping input names to arrays, a list of input arrays, or a single input array

• outputs (Optional[List[str]] = None) – The outputs of the inference to return, if None all will be returned.

• model_version (Optional[str]) – The version of the model to use.

Returns

Results of inference

Return type

Dict[str, np.ndarray]

Examples

>>> client.get_inference("s3://models/my_model_key.onnx", np.float32(np.random.random((1, 3, 10, 10))), outputs=["loc", "landms"], model_version="v1")

litecow.server module

class litecow.server.ICOWServicer(batch_hint: int, aws_access_key: str, aws_secret_key: str, endpoint_url: str)

Bases: litecow.common.litecow_pb2_grpc.ICOWServicer

get_inference(request, context)

Handles requests for service get_inference calls.

Parameters

• request (litecow_common.litecow_pb2.InferenceRequest) – The InferenceRequest to answer.

• context (grpc.ServicerContext) – The context for this request

Returns

Outputs of inference encoded as a NamedArrays

Return type

Union[litecow_common.litecow_pb2.NamedArrays, litecow_common.litecow_pb2.ArrayList]

litecow.server.main()

Run the icow_service with the arguments provided on the command line.

Uses arguments provided on the command line and environment variables set on the system to run an icow_service. Must set environment variables: AWS_ACCESS_KEY AWS_SECRET_KEY, S3ENDPOINT_URL to values for the s3 storage so the icow_service can access s3.

litecow.server.serve(workers: int, port: int, batch_hint: int, aws_access_key: str, aws_secret_key: str, aws_endpoint_url: str) → None

Run the ICOWServicer with arguments passed.

Parameters

• workers (int) – The number of max workers for the ThreadPoolExecutor running the server.

• port (int) – The port number to bind the service to.

• batch_hint (int) – The initial number of inputs to batch together in a single forward pass.

• aws_access_key (str) – The aws access key for the s3 storage to be used by the service.

• aws_secret_key (str) – The aws secret key for the s3 storage to be used by the service.

• aws_endpoint_url (str) – The url for the s3 storage used by the service includes scheme and port. Like http://my-s3-service.us:1337

Module contents