Skip to main content
Artificial Intelligence

PalmClaw: A Native On-Device Agent Framework for Mobile Phones

| Source: arXiv

Preprint — not peer-reviewed. Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users' data, sensors, and daily-use applications. Existing mobile agents mainly operate smartphones through graphical user

Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users' data, sensors, and daily-use applications. Existing mobile agents mainly operate smartphones through graphical user interface (GUI) actions such as tapping, swiping, and typing, which often form long, interface-dependent sequences, cannot directly access device capabilities, and make execution boundaries difficult to define. We present \textbf{PalmClaw}, an open-source agent framework that runs natively on mobile phones and manages the sessions, memory, skills, tools, and agent loop directly on the device. PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. This design enables agents to use mobile capabilities directly while keeping each action explicit and controlled. Experiments show an 11.5\% relative improvement in task success and a 94.9\% reduction in completion time over the strongest baseline, with lower setup burden and traces illustrating how execution boundaries are applied. Code is available at https://github.com/ModalityDance/PalmClaw.

Read the original source →

Related Stories

Artificial Intelligence

Deep learning of fossil pollen morphology reveals 25,000 y of ecological change in eastern African grasslands.

Grass (Poaceae) pollen is largely overlooked in investigations of grassland evolution because the pollen of most species cannot be differentiated using traditional optical microscopy. However, the combination of superresolution microscopy and deep learning enables the capture and quantification of distinct morphological variation across the pollen of grass species. Using a semisupervised deep-learning strategy, we trained convolutional neural networks (CNNs) on superresolution images of known mo

Continue reading
Artificial Intelligence

Tyrosine phosphorylation unfolds nucleophosmin and disrupts its integration into the nucleolus.

Nucleophosmin (NPM1) is a multifunctional nucleolar protein essential for ribosome biogenesis, genome stability, and stress responses. Its integration into the nucleolus depends on its oligomerization and multivalent interactions that enable liquid-liquid phase separation (LLPS). Here, we investigate how tyrosine phosphorylation at Tyr17, Tyr29, and Tyr67 located within the interface between monomers at the N-terminal oligomerization domain regulates NPM1 structure and function. Replacing tyrosi

Continue reading
Artificial Intelligence

Shared patterns of human milk composition link mammary gland function to infant growth.

The mammary gland produces nutrient-rich human milk (HM), yet how mammary functional state shapes HM composition and infant outcomes remains poorly understood. We used HM multi-omics - metabolomics, proteomics, micronutrients, macronutrients, and HM oligosaccharides - across 1,543 samples from three cohorts, including two randomized trials, as a noninvasive readout of mammary functional state. Trajectory modeling and multi-omic integration showed that maternal supplementation improved recovery f

Continue reading
Artificial Intelligence

Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study.

Artificial intelligence (AI)-based systems show promise for assisting primary care providers (PCPs) with patient care. We aimed to evaluate the safety and quality of clinical conversations of a patient-facing conversational AI system, which engaged in real-world urgent primary care appointments. In this prospective, single-centre, single-arm feasibility study, English-speaking patients aged at least 18 years interacted with the Articulate Medical Intelligence Explorer (AMIE) up to 5 days before

Continue reading
Artificial Intelligence

Large-scale semidefinite programming with graphics processing units.

Semidefinite programming (SDP) provides a powerful framework in applied mathematics with applications spanning optimization, machine learning, quantum computing, and beyond. However, the computational cost of solving large-scale SDP problems remains a significant practical limitation. We break this long-standing computational bottleneck through a synergistic codesign of low-rank algorithms and graphics processing unit (GPU) architectures, developing accelerated first-order methods that leverage

Continue reading