[FEEDBACK WANTED] Regarding Jupyternaut

[dlqqq]

Problem

We've made major changes to the jupyter-ai-jupyternaut experience to support Jupyter AI v3.1.0 on a best-effort basis -- but acknowledge that it is rough around the edges and that we need more feedback before we mark it as a stable (albeit optional) feature.

It has been challenging to support this package because maintaining an entirely custom agent harness is expensive. The open source AI library landscape is shifting constantly, and there is a continuous need to keep this package updated. We also have very limited feedback on how and why users still use Jupyternaut; we merely understand that there exist users who want to stick with using Jupyternaut, and that there are many who still consider it an integral part of their Jupyter AI experience.

If you use Jupyternaut, please try the new jupyter-ai-jupyternaut==0.1.0b0 experience, and let us know what you think! We're particularly interested in getting the community's take on these questions:

1. Does the new experience work for you? Are there any major regressions or bugs that break your workflow?

2. What makes you use Jupyternaut instead of built-in agents? e.g. Is it too difficult to configure custom models using existing harnesses like OpenCode or Goose?

3. Would being able to create custom AI personas using the same LiteLLM + LangChain stack (Jupyter AI v3.2 proposal) meet your needs? If not, why not?

Proposed Solution

• Figure out the story around Jupyternaut. Our proposal is that the future of Jupyternaut should be centered around what the community wants from it:

  • If people don't see a need for the existing LangChain + LiteLLM harness vendored in Jupyternaut, we can fully deprecate it after Jupyter AI v3.2.0 makes it easier to make custom AI personas connected to custom models/agents at different endpoints.

  • Otherwise, if people want to still be able to re-use the LiteLLM + LangChain stack it is built on, it should be vendored as an optional persona engine (e.g. jupyter-ai-lllc-engine). Ownership of this new package should be transferred to new maintainers who are interested in maintaining it, if they exist.

  • Jupyternaut could stay as the 'frontend-only' AI persona for Jupyterlite use-cases, which we plan to support in the future. Of course, the implementation would be completely different, so maybe it's better to track that in a new git repo to reflect that.

• Update the Jupyternaut docs and screenshots once experience is stable in Jupyter AI v3.1.

[dlqqq]

TODO : mention these users after v3.1.0 is published.

cc @joequant @minrk 

[mwuttke]

Hello David,

thanks for opening a discussion about Jupyter AI & Jupyternaut.

Here is a feedback from a managed multi-tenant deployment, in case it's useful for the decision.

Context: We run JupyterHub on Kubernetes at an University in Berlin with a dedicated
"AI" profile. JupyterLab is built from quay.io/jupyter/datascience-notebook,
Jupyter AI is installed and configured at image build time via
/etc/jupyter/jupyter_server_config.py. The backing model is a self-hosted
Qwen3-Coder served by vLLM behind an internal OpenAI-compatible endpoint,
reached through Jupyternaut's LiteLLM layer with model_parameters supplying api_base.
End users do not configure anything; they open the chat and it works.

1. Does the new experience work for you?

One thing worth flagging independently of Jupyternaut's future: the in-chat model picker
added in 3.1 advertises the entire LiteLLM chat catalog
(JupyternautPersona._build_model_configuration), and the existing
allowed_providers / allowed_models traits are not enforced against it. For managed
deployments this is a meaningful regression in administrative control.

2. What makes you use Jupyternaut instead of built-in agents?

It isn't a preference for Jupyternaut as an agent — it's the only persona in the stack
that we can point at our own inference endpoint through central, image-level
configuration.

Concretely, what our situation requires:

• A self-hosted OpenAI-compatible endpoint. Our institution cannot send user code or
  data to third-party frontier providers. Every ACP persona currently shipped assumes an
  external service plus per-user credentials.
• Zero user-side configuration. Several hundred students and staff share one image.
  Anything that requires each user to authenticate, install a CLI, or edit an
  agent-specific settings file is not deployable for us.
• A declarative config file baked into a container image. model_parameters with
  api_base in jupyter_server_config.py is exactly the right shape: one file, applied
  identically to every pod, reviewable in Git.
• No extra runtime in the image. The ACP agents bring a Node runtime and agent
  binaries; that's additional supply-chain surface we'd rather not ship to every user pod.

So the answer to "is it too difficult to configure custom models using OpenCode or
Goose?" is: it's less about difficulty and more that their configuration model is
per-user and per-agent, whereas ours has to be per-image and central.

3. Would custom personas on the LiteLLM + LangChain stack (3.2) meet your needs?

Yes — that would cover us well, and honestly better than Jupyternaut does today, provided
two things hold:

1. The LiteLLM + LangChain engine survives in some form. Option 2 in your proposal
   (jupyter-ai-lllc-engine) is what we need. PersonaModel.model_id +
   PersonaModel.model_url maps cleanly onto what we express today as
   model_parameters / api_base, and defining our institutional persona once as a
   .persona.md shipped in the image would be a clear improvement over the current
   traitlet configuration.

2. There is an administrator-level way to constrain what personas users can create.
   This is the part that currently worries me about the 3.2 design. .persona.md files
   are read from .jupyter/personas, which is user-writable. As proposed, any user can
   define a persona with an arbitrary model_id, an arbitrary model_url, and an
   arbitrary engine, and load it with /refresh-personas. For us that means a
   deployment-wide policy ("this installation talks to exactly one endpoint") is not
   expressible anywhere in Jupyter AI.

   I understand — and fully accept — that this is not a security boundary: a user with a
   terminal in the pod can reach any endpoint the network permits, and we enforce the
   actual constraint with a Kubernetes NetworkPolicy. But an administrative control is
   still valuable for defaults, for institutional data-protection documentation, and for
   avoiding confusing failure modes where a user selects a model the deployment cannot
   reach. Something like an allowed_engines / allowed_models traitlet honoured by the
   persona manager, and/or a way to restrict persona definitions to admin-provided paths,
   would close this.

If a smaller LiteLLM engine package lands, we're happy to test it in a real multi-user
deployment and report back.

Thanks & Greetings,
Michael

[joequant]

1. Jupyternaut is not working for me. I just get an empty chat screen

2. What makes me use Jupyternaut is that I just want to have a chat with my local LLM and my local openclaw/hermes set up. This should be just a matter of taking the URL to an openai/ollama interface and just having it work, and jupyternaut is the only way that I can see where I can just connect a URL to an custom agent. The fact that Jupyternaut contains a ton of custom harness code is why it is necessary, because that code has to exist somewhere.

So typical work flow, I download a new model from huggingface and run my stuff against the new model. I should be able to do this in less than a minute.

3. The other thing that is a must have (which Jupyternaut doesn't have) is the ability to set up multiple personas that are connected to different LLM's. One reason I am trying to get this work with ACP's, is that it would be fine if there was a well documented way of creating personas, but there isn't. One reason I've ended up trying to work through Jupyternaut is that because the code exists, I can see what needs to be modified to create a given workflow. This isn't true for the non-Jupyternaut mechanism

[joequant]

This is a hard requirement for us as well. One thing to note is that educational/research institutions
are under strict legal requirements, as well strict internal process requirements. Unpublished, unreviewed
data cannot be sent or processed off-site, full stop.

There are also very strict privacy requirements. For example, if I am grading papers the mark that a student
makes is subject to strict confidentiality requirements, and I cannot send this data off-site without legal risk.

• A self-hosted OpenAI-compatible endpoint. Our institution cannot send user code or
  data to third-party frontier providers. Every ACP persona currently shipped assumes an
  external service plus per-user credentials.

[joequant]

I've bundled together JupyterLab with Openwebui at https://github.com/hkphysics/ai-server

What I've done is to move a lot of the access control out of Jupyter and Openwebui into litellm.

Also to echo @mwuttke, the model system that OpenAI and OpenCode uses is designed for invididual users and not workgroups. So everything ends up being user based, which makes centralization at the workgroup level very painful.

[joequant]

Also one thing that you should consider is the release process. The thing is that right now Jupyternaut is not working at all for me (I don't get a selector). This is going to cause huge problems for user takeup, since people won't use a platform that just breaks.

[joequant]

Also about my context. I have three different Jupyter instances, one for family, one for the startup that I am co-founding, and one for my academic work. These three instances have completely different data stores, run on different machines, but I can quickly set up a new system from hkphysics/ai-server. However these three instances call litellm and are running a central ollama instance, so they share a GPU resource.

[srdas]

@joequant I just installed everything using the attached script for the dev install install.sh, would you try this and see if it works?

install.sh

1. Download install.sh
2. chmod a+x install.sh
3. ./install.sh
4. Just respond to all the prompts to install what you need.
5. At the end of the script, change to the jupyter-ai-devrepo folder: cd jupyter-ai-devrepo
6. Activate the environment: source .venv/bin/activate
7. Kick of Jupyter: just start

Hope this helps.

I'll look into the user install first thing next week.

[joequant]

Will try this. Also I wrote down in #1624 with my latest thoughts as to the role of Jupyternaut. The thing about Jupyternaut is that it contains a lot of random code that's there so that it just works. That's both good and bad, but we really want it so that all of the random code to do connections are in one place.

The other thing is that I've been done a lot of development work and some of the things that I thought were bugs turned out to be old versions, or development artifacts. So a "WORKSFORME" is very useful since it tells me that there is something odd in my setup.

[joequant]

One thing I do think is a problem with jupyternaut is that it contains litellm code embeded. This results on awkward situations when you just have an external litellm system. One thing to look at is the way that openwebui handles connections. They have two interfaces, an openai interface and a ollama interface, and once you connect to the interface then it tells you what models are available.

One possibility is that instead of having Jupyternaut settings, you have a "persona generation" settings. So Jupyternaut would be spilt into an abstract base class, and a small layer on top. If you want to create another connection, then rather than change the Jupyternaut settings, you would create a new persona with the alternative settings.

[joequant]

One thing that you'd want is to have the user be able to rapidly switch between models, and it would make more sense to have different models each have a different persona than to make Jupyternaut multimodel. The other thing is that at least for me, having a GUI is not top priority, and I am fine with editing configuration files as long as its documented what the configuration file is.

[srdas]

This is a good idea, I.e., be able to spawn a new Jupyternaut-based persona with preset model and settings, so a user can create multiple such personas for different types of usage.