✍️
Today IL
  • Today I learned!
  • Deployment
    • Rolling, Canary, Blue-green deployments
    • Kubernetees Helm Charts
  • AI/ML
    • SeldonIO
    • Installing software in E2E cloud compute
    • Watching nvidia-smi
    • How does github copilot works?
    • composer library
    • Better to pass callback in fit_one_cycle
    • Eliza - demo
    • Helsinki-NLP Translation models
  • Fastai Learning
  • Python
    • Understanding get_image_files in fastai
    • Resizing an image to smaller size
    • Extracting a Json Object as List(regex)
    • f-strings debugging shortcut
    • Pytest
    • conda switch between python versions
    • Nested functions exception handling
  • Programming
    • Installing Linux Operating system
    • Certbots usage
    • Code highlighting in Google Docs
    • HTTP Methods
    • How to use vertical mouse?
    • HTTP Status Codes
    • Keycloak, Oauth, OpenID connect, SAML
    • Why should NPM packages be as small as possible?
    • Clean Architecture
    • REST vs gRPC
    • Keycloak concepts
    • what is proxy server and nginx?
    • Asymptotic Time Complexity
  • async/await
    • JavaScript Asynchronous operation
    • Lesson 2- Eventloops
    • Lesson 1- asyncio history
    • Lesson 3- using coroutines
    • Lesson 4- coroutines in hood
    • Python async/await
    • JavaScript
  • R Programming language
    • Facet_grid and Facet_wrap
    • geom_point
  • C programming language
    • Inputting String in C programming language
    • Checking if a element is passed as input or not?
  • Git/Github
    • give credits to other people
    • one time setting to easily use Github
    • Checkout to specific tag
    • git suggestions in PR
    • Using emojis in git commits
  • Databases
    • Postgres Database Dockercompose
    • TIL New SQL Operators - Except, UNION, Distinct
    • Analysing Performance of DB Queries
    • Querying Date ranges in postgres
    • Handling Database disconnects in SQLAlchemy
  • WITH NO EXCEPT
  • What is difference with JSON documents in postgres and Mongodb
Powered by GitBook
On this page

Was this helpful?

  1. AI/ML

SeldonIO

PreviousAI/MLNextInstalling software in E2E cloud compute

Last updated 3 years ago

Was this helpful?

Seldon core converts your ML models (Tensorflow, Pytorch, H2o, etc.) or language wrappers (Python, Java, etc.) into production REST/GRPC microservices.

Seldon handles scaling to thousands of production machine learning models and provides advanced machine learning capabilities out of the box including Advanced Metrics, Request Logging, Explainers, Outlier Detectors, A/B Tests, Canaries and more.

Deploy your model using pre-packaged model servers

We provide optimized model servers for some of the most popular Deep Learning and Machine Learning frameworks that allow you to deploy your trained model binaries/weights without having to containerize or modify them.

Send API requests to your deployed model

Every model deployed exposes a standardised User Interface to send requests using our OpenAPI schema.

This can be accessed through the endpoint http://<ingress_url>/seldon/<namespace>/<model-name>/api/v1.0/doc/ which will allow you to send requests directly through your browser

These are Seldon Core main components:

  • Reusable and non-reusable

  • to containerise models

  • CRD and

  • for advanced inference graphs

as well as integration with third-party systems:

  • Kubernetes Ingress integration with and

  • with

  • with

  • with

To learn more about Seldon refer below links:

  • https://docs.seldon.io/projects/seldon-core/en/latest/index.html\

  • docs.seldon.io

  • https://github.com/SeldonIO/seldon-core\

  • https://github.com/SeldonIO/MLServer\

What is difference b/w Seldon Core and MLServer?

Basically Seldon Core is the "orchestrator" that takes your model, deploys it on to a server, collects metrics and logs and creates services and routing over the top. MLServer is the "engine" that actually handles executions of that model during inference. Seldon Core uses MLServer as a runtime to deploy your models on.

Another big difference is seldon-io core uses Flask under it's backend (for non-parallelization).

While ML-server is build with FastAPI, so our APIs with seldon can work on top of it as well.

¶
¶
model servers
Language Wrappers
SeldonDeployment
Seldon Core Operator
Service Orchestrator
Ambassador
Istio
Metrics
Prometheus
Tracing
Jaeger
Endpoint Documentation
OpenApi