I already filed a draft proposal to Invent KDE, but I feel it needs refinement, so I am turning to the collaborative wisdom of this wonderful community.
Plasma already handles a lot of individual sources of information – documents, calendars, contacts, bookmarks, source URLs, file tags, file metadata … but they are not working for the user in a coherent, orchestrated manner. Instead, they are essentially silos.
So the main idea is to that Plasma gets “smart” and finds a way to bind this data together and add meaning to it. Combining/linking individual information could contextualise it to make the information much more useful and organic for humans to use.
As an example, a human knows that the following are all the same person:
@hook in the address book and all the contacts (e-mail, XMPP, Matrix, blog, physical address, …)
Currently, to Plasma those are all completely separate things.
We also live in a time, where information is very often cross-referenced, either through hyperlinks or otherwise. Many people rely on so-called knowledge management systems like Joplin, Obsidian etc., and from what I can tell KDE/Plasma is not far from achieving the same on a desktop-level by better leveraging the features it already has (Baloo, Akonadi, file tags, …) and improving it with more connections and some UI.
I imagine a world where Plasma would understand such and similar connections.
Whether that would mean adding (a) new application(s) or “just” improving Plasma, I think it is up for debate what is the best approach from the users’ – incl. your! – perspective.
First of all, I propose that this “Smart Plasma” should not** need** any LLM to work. (Personally, I don’t intend on using it with LLMs.)
But I also know that some KDE users use or want to use AI/LLM. And for those people, it would be best – and in line with KDE’s spirit – that Plasma enables a use that is privacy respecting and energy efficient. As such, a small local LLM/AI would be a good choice, but those work best if they can connect to structured data (see e.g. Retrieval-augmented generation - Wikipedia ). And this is what “Smart Plasma” would provide.
Please, let’s not forget, that if Plasma is easier to grasp for normal humans, it would also make it more accessible. Especially for people who may rely on computer assistants, such as people with disabilities and limitations.
Next steps
Please share your ideas, aspirations, concerns, suggestions etc. so we can create a good proposal together.
I think Plasma has great potential to be smart and be (even) more approachable to users
It is worth pointing out that AI does not begin and end with LLM and resource hungry data centers that need to raze a forest to produce a stupid image of your uncle riding a giraffe.
AI has been around and helping engineers and common folk since the 60s. The opponent you play against on you phone’s chess game? AI. The software that can recognise text in images? AI. Expert systems that figure out if a bridge will collapse or not? AI.
None of the above uses LLM technologies and are frugal in their energetic requirements.
It may be worth making this point clear in your proposal.
And for those people, it would be best – and in line with KDE’s spirit – that Plasma enables a use that is privacy respecting and energy efficient.
And for a clear conscience, it would be best – and in line with KDE’s spirit – that Plasma does not bake in means of generative slop production. It would be best, and in line with KDE’s spirit, not to further enable the antisocial techbro grift of “”“”“AI”“”“”.
“those people” can find themselves third-party integrations or vibe code their own thing if they want slop in their desktop so badly.
And TBH, KDE is not entirely bereft of AI (not in the genAI/LLM way). In DigiKam you could use local face-detection that you taught on your photos already for many years. And then there was the whole promise of Mycroft (now OpenVoiceOS) being an optional voice assisstant for Plasma.
The casual aibro-placating aside, you should be more specific as to how exactly the desktop is meant to connect the data points you mention in any useful (and not counterproductive) way. Especially if plopping down an “agent” is ruled out, this is too vague to be actionable.
When an OS tries to be “smart” you usually get lots of unpredictable behaviour and a privacy & security nightmare’s worth of metadata, both hard to tame. People run in horror of Windows 11 for this reason. What execution on this idea do you have in mind that would keep out of the user’s way so as to remain a net positive?
Do not conflate non-generative machine learning and generative “AI”, no gripes were to be had with the former before this current bubble and it remains just fine now, for the purposes of a productive conversation it is not “AI”
I fully agree, but have the unfortunate bias that I was using NEPOMUK/Strigi before, so in my mind it is something between what NEPOMUK did, how (e.g.) Obsidian handles cross-references and knowledge maps and what Scott Jenson described in the video posted above. I can come up with some use cases, but they will be either very oldskool (i.e. was already state of the art 20 years ago) or completely outlandish (I’m not a coder really, nor a UX expert, just a user).
That is why I’m asking the general KDE community for ideas what they imagine a smart desktop would look like.
Ultimately, the goal should be to make the data connected and as such more useful.
The final version has been greatly changed, particularly with a lot of very useful ideas from @osezer (hat tip to @ngompa for the connection too!).
So please re-read the final proposal and if you like it, you are more than welcome to volunteer for any of the roles to help out achieving this goal.
I think we’re on to something here and it would be a pity not to push KDE/Plasma to evolve and surpass other desktops.
For your convenience, I am copy-pasting the final proposal’s text below:
Description
The goal is to make your computer understand how your files, contacts, and tasks are connected, saving you time and effort.
The burden of keeping the location and connections between different pieces of information (content of documents, notes, pictures, calendars, contacts, bookmarks, conversations, tags, …) should shift from the human to the machine. Plasma should do the heavy lifting of the menial data management, and present it to the user so that the human can let their creative juices flow.
Core Ideas:
Learning from Your Habits: Instead of forcing you to organize everything manually, your computer can naturally learn from what you do.
Smart Activities: KDE Plasma has a feature called “Activities” to group your work environments. If you search for something while in the “Work” Activity, the system will automatically prioritize your work files and contacts.
Asking the Right Tool: If you search for a photo using KRunner, it won’t try to scan your photos itself. It will instantly ask your photo app and Baloo for the answer, pulling the best results together.
Modularity: Instead of relying on a single heavy database, we want to use small, smart, mostly pre-existing tools that talk to each other smoothly, while keeping everything local, safe and private. That way the user can tailor the experience to their liking.
Voice Commands: Instead of sending your voice to a cloud server, a tiny helper on your machine can translate your spoken command directly into a system action.
Auto-Organizing: A small helper could quietly label your saved screenshots as “receipts” or “notes” in the background so you can find them easily later.
Playing Nice with Others: We will provide standard ways for external apps to ask the system simple questions without interfering with the core of your computer’s system.
Enabling More Creative Workflows: While not directly in scope of this goal, a knock-on effect of contextualising and exposing rich data could be that more creative workflows for different tasks or even desktop concepts emerge down the line.
What will it take
Here are a few architectural approaches we could explore:
We can achieve deep contextual awareness purely through deterministic algorithms and statistical heuristics:
Activity-Driven Dynamic Ranking: KActivities already isolate workflows. Baloo and KRunner can dynamically adjust search scoring based on the active Activity, prioritizing code repositories in a “Development” activity, and slide decks or calendar items in a “Work” activity.
Temporal Co-Occurrence Graphs: Rather than asking users to manually tag relationships, Plasma can observe temporal patterns. If a document, a web URL, and an address book contact are routinely opened within the same working session, Plasma can establish an implicit “soft link.” Searching for one naturally surfaces the others as related context.
Smart Clipboard & Seamless Vision: KDE’s screenshot tool, Spectacle, already features Optical Character Recognition (OCR) to extract text from images. We can deeply integrate this capability into Klipper (the Plasma clipboard). By combining OCR with pattern recognition, the system can instantly identify when you copy a URL, an event date, or a tracking number even directly from an image capture and automatically offer quick actions for your browser or calendar without requiring heavy AI and using simpler ML technics or models to handle it. Another idea to contextualise copied content is a clipboard “file” / scrapbook that is associated with a specific Activity or even a specific document.
Federated Search & Query Routing
Instead of trying to centralize all desktop metadata into one giant master index, Plasma should rely on a federated query architecture.
On-the-Fly Metadata Correlation: Data stays where it naturally belongs (Akonadi for PIM, Baloo for file indexing, browser profiles for web history). When a search is performed in KRunner, it acts as a federated coordinator. If a document’s metadata contains an author name, KRunner correlates that author against Akonadi contacts on the fly, presenting unified results without requiring a persistent master database.
Leveraging Domain-Specific Indices (e.g., digiKam): Individual KDE applications already implement powerful local discovery. For instance, digiKam provides local semantic and similarity search for photos. Plasma can query digiKam’s local index through standard interfaces instead of re-indexing image content at the shell level.
Specialized, Low-Resource Edge ML
Where machine learning adds clear value, it should be limited to small, specialized, on-device models running via lightweight runtimes (such as ONNX Runtime or LiteRT or ncnn).
Offline Voice & System Intention (e.g. Kelam): Building upon offline dictation tools like Kelam (which uses local Whisper models for speech-to-text), Plasma can map natural spoken commands to desktop actions. A compact, local intent classifier translates spoken phrases directly into existing Plasma or KWin D-Bus calls entirely on-device.
Background Automated Tagging: Low-priority background workers can process incoming downloads or screenshots with tiny, quantized vision models, applying basic semantic labels (e.g., “invoice”, “receipt”, “screenshot”) to file attributes so Baloo can index them instantly.
Standardized Desktop Action Interfaces
For users running local AI tools or custom scripts (such as Alpaka for local LLMs), Plasma can provide clean, structured access to system capabilities:
Introspectable D-Bus Action Registry: Expose well-defined D-Bus methods for common desktop actions (querying active windows, setting focus, managing notifications, retrieving contextual file lists).
Pluggable Runner Extensions: Allow external local clients to act as bidirectional KRunner providers or consumers, enabling local models to query desktop context and trigger system actions cleanly and safely.
If we rely on our existing Inter-Process Communication (IPC) and federated indexing, we can shift the burden of data management away from the user while keeping the desktop responsive and entirely private.
Human-centric UX
All the backend intelligence in the world is useless if the interface only makes sense to developers – a trap that previous semantic desktop efforts often fell into. The user-facing side must be organic, intuitive, and entirely frictionless:
Natural Language Interaction: Users should be able to type or speak exactly what they mean into KRunner (e.g. “Find the PDF John sent me yesterday”) without needing to memorize complex search operators or navigate rigid database structures.
Contextual Surfacing: Instead of dumping a raw, unorganized list of files, search results and system prompts should be grouped by context (e.g, “Files related to my upcoming meetings this week”) and presenting meaningful context (e.g. immediate snippets/previews of what was searched), offering the user a cohesive working set rather than scattered data points.
Invisible Assistance: The UX should not force the user into a separate “data management” application. The smart features should live directly inside the tools they already use (like the clipboard or screenshot tool), offering one-click actions exactly when they are needed.
How we know we succeed
Different KDE (and other) data silos will be connected, resulting in context-richer information and knowledge graphs.
Users will find it much more natural to interact with Plasma and the information they have at hand, finding what they are looking for not just faster, but in a manner that invites unburdened creativity. Instead of spending time and effort manually sorting, tagging and searching via separate apps (plus fighting against bugs), user will be able to rely on Plasma to simply handle their data.
As a very important side-goal, users who rely on voice commands – whether due disability or otherwise – would have a much better time interacting with their computer through Plasma. This is an important improvement of accessibility.
Several (old) Baloo, Akonadi etc. bugs get triaged and fixed.
Kdenlive: The 25.04.0 release integrated powerful local tools like Whisper (for audio) and SAM2 (for video tracking) to process media directly on your machine. Kdenlive 25.04.0 released - Kdenlive
Kelam: This tool allows you to speak to your computer and dictate text across the system. It works completely offline. Onuralp SEZER / Kelam · GitLab
Alpaka: A clean interface that lets you talk to local AI models directly on your desktop, proving we can have smart assistants without relying on the cloud. Utilities / Alpaka · GitLab
Recoll: An alternative to Baloo, which indexes also some non-KDE app data and provides previews/snippets on search – good for inspiration: https://recoll.org/
FAISS: A library for efficient similarity search and clustering of dense vectors: https://faiss.ai/
Scott Jenson: How the Desktop UX needs to evolve to keep up with Local-first: a great historic overview and exploration of different UX improvements of how richer data and local models can bring a breakthrough in desktops and their usability: https://www.youtube.com/watch?v=-IOLRcFC6OY
From what I understand, a lot of such history is already stored on the system (e.g. by KActivities, Baloo, Akonadi, filesystem, shell), but we want to make better use of it. Whatever additional pieces of information would help here, should definitely be optional, so the user can enable or disable them individually (see “Modularity”).
Just to clarify, this history should not leave your computer.
@osezer is much more knowledgeable than me here, so I hope he can explain this in more detail.
I know there’s no intention for it to leave the computer. But one question I would anticipate some people asking here is “how do I make sure it isn’t even stored, so that no malware could possibly exfiltrate it?”
For example you’ll find threads here where users are concerned with preventing clipboard history being stored, and similarly (though it’s not KDE-specific) there are people who want to limit their shell history.
It might be worth mentioning that in the “Modularity” paragraph? i.e. be explicit about opting out of features as well as “tailoring” them.
Very fair point and definitely something we need to consider.
In my mind, of course, a user should be able to run Plasma with all this disabled, if that is what they want. Plasma should continue to work as a “normal dumb old” desktop for them.
Hello @pg-tips let me try to answer couple of questions I see in forum based on questions;
No Central Database to Exfiltrate: The proposal explicitly avoids creating a new, vulnerable master database of the user’s habits. Under the Federated Search & Query Routing section, it states that Data stays where it naturally belongs. When a user searches, KRunner performs “On-the-Fly Metadata Correlation or Embedding search”. This is on the fly data and not permanently saved, there is no centralized graph for malware to steal.
Modularity Means Total Control: A core idea of the proposal is also Modularity. It advocates for using “small, smart, mostly pre-existing tools” so that “the user can tailor the experience to their liking.” Because the intelligence is decentralized, a security-conscious user can simply opt-out of or disable specific components (like the Smart Clipboard or background tagging) ensuring that specific data is never processed or stored in the first place. Also even in pass we still control
Activity-Based Isolation: The architecture leans heavily on Activity-Driven Dynamic Ranking and the fact that “KActivities already isolate workflows.” Users can leverage Plasma’s existing Activity privacy settings to create strict boundaries. For example, a user could have a general “Work” activity where temporal patterns are observed, but seamlessly switch to a “Private” activity where history and context tracking are completely disabled at the system level.
Also all of those features are can be hopefully enabled or disabled (depends on compute or usage requirements because we don’t make users to suffer in their desktop performance. For example some of the existing tools in proposal (for example one of the recent digikam feature, uses model that require certain compute requirements) Some of them can be default enabled or permanent (I am not saying it is going to be just idea) Of course many factory are important such as “is it faster” or “is it efficient” and other cases.
This partially reminds me of the Windows search framework into which various apps could hook into and extend in various ways. Baloo used to be like this too. I don’t recall why the PIM indexing was moved out.
Also KRecentDocument sounds like something with similar requirements - cross toolkit interop.
That kind of plugin system would be needed for non-kde apps, as IMO you essentially can’t do this without apps feeding the information into the system.
Stop right there, Plasma does not owe sloperators anything. People do not like AI slop, and people will not appreciate KDE comitting to doing things for the sake of accomodating slop.
(It would be very disappointing to see a version of KDE so morally bankrupt as to accept “think of the AI users” as a framing for why something should be done.)
At the risk of agreeing with hsza here, how dare I after all (To be clear, I’m joking), I am generally opposed to any AI by default or similar supposed “smart” things. I absolutely resent things (like Windows and like my TV) that think they’re clever, worse when they think they’re cleverer than I am and know better than me what I do and don’t want.
I also am concerned, as pointed out previously, about the increased risk to users of having a compiled database of information connections that can be lifted by malicious actors, especially with the frequent reports of security exploits by improperly locked down AI. It is true that every convenience is a trade-off somewhere against security but at least to me as an organised person this does not qualify is a valid trade.
Now, that’s not to say that KDE should not do anything to make it easier for people who want to install “smart” infrastructure for themselves but I strongly believe that’s where the line should be drawn - opt-in only.
I’m glad to see the clarification between AI, LLMs and ML, I must apologise for using an overly broad term - I really should adjust to this at some point. Using ML is preferable to LLMs or agentic AI though it still doesn’t help with the disdain for things that think themselves clever.
Unfortunately though having read more of that GitLab discussion I see that I am late to the party regardless so I shall wish you the best of luck and hope if it’s voted through that there is a readily available “Off” switch, preferably set “Off” by default.
