zhangyi1357 df5d21e49c
feat: aimrt_py support ros2 protocol interface (#95)
* feat: add checks for ROS 2 message type support

Implement validation functions to ensure message or service types are compliant with ROS 2 standards, enhancing compatibility and error handling when integrating ROS 1 and ROS 2 environments.

* build: update file globbing to include configure dependencies

Ensure source files are automatically detected during configuration by adding CONFIGURE_DEPENDS to the file globbing commands for C++ and Python files. This improves build reliability as it accounts for changes in the file structure.

* feat: add ROS2 message support and enhance publishing functionality

Implement functions to register, publish, and subscribe to ROS2 messages, improving interoperability with ROS2 systems. Adjust type support to handle both protobuf and ROS2 message types, streamlining the process of message serialization and ensuring robustness in message handling.

* feat: add HTTP and ROS2 example configurations and applications

Introduce example configurations and applications for both HTTP and ROS2 channels, enabling greater flexibility and showcasing the capabilities of the system for publishing and subscribing to messages.

* chore: disable unimplemented copy and move functions

Throw runtime errors for the unimplemented copy and move methods to ensure clarity on their current status and prevent accidental usage.

* feat: enhance message publishing with iteration and logging

Implement continuous publishing of messages with incremental identifiers in the publisher application. Update the subscriber to log message count, improving visibility into received messages.

* refactor: reorganize and encapsulate ROS2 message handling functionality

Streamline the handling of ROS2 messages by encapsulating related functionality into a new utility header file. This enhances code clarity and maintainability while adhering to best practices in modular design.

* refactor: streamline protobuf and ROS2 message publishing

Enhance context handling and unify publish type registration for better clarity and usability. Simplify serialization type management and improve the architecture for future extensions, particularly for messaging features.

* refactor: streamline messaging functions

Enhance message publishing by consolidating function calls and improving clarity. Update context handling to better support message serialization types, allowing for a more flexible interaction with both protobuf and ROS2 messages.

* refactor: simplify message subscription handling and reduce publish frequency

Adjust the message subscription method to handle both Protobuf and ROS2 message types more efficiently and eliminate redundant subscription functions. Decrease the publishing frequency in the example app for improved performance and readability.

* fix: remove unnecessary error message for ROS 1 type detection

Eliminate confusion by removing the printed warning about ROS 1 message types, as it was not providing significant value in handling unsupported types.

* docs: update README and add ros2 channel examples

Clarify publisher behavior in examples and provide comprehensive documentation for ros2 channel implementations.

* feat: support ros2 message types in aimrt_py channel

Add support for ros2 message types in the aimrt_py channel to enhance interoperability with ROS2-based applications.

* fix: improve error handling during ROS2 message serialization and deserialization

Enhance robustness by adding exception handling to ensure the process correctly handles serialization and deserialization failures, improving overall stability in handling ROS2 messages.

* fix: correct argument validation and formatting

Remove unnecessary whitespace and enhance error handling for argument types in the publishing function, ensuring clear type checking and preventing potential runtime errors.

* refactor: streamline message conversion logic

Simplify the conversion of ROS2 messages to Python objects by moving the conversion logic inline. This eliminates unnecessary function calls and improves performance while maintaining clarity in the subscription method.

* feat: add ROS2 support to Python runtime

Enable linking and compilation of ROS2 interfaces based on the build configuration. This improves flexibility by allowing the Python runtime to interface with ROS2 when needed, enhancing interoperability and feature set.

* feat: enhance message type handling for publishers and subscribers

Add utility functions to generate names for ROS2 and protobuf message types, centralizing logic for type name creation. Update the publisher and subscriber to utilize these functions, improving code readability and maintainability while ensuring accurate message type management.

* feat: update message types to use RosTestMsg for better compatibility

Register new message type RosTestMsg in both publisher and subscriber applications. Update data format for publishing events to improve clarity and consistency in message handling. Ensure proper setup in the shell scripts for localization configuration.

* chore: consolidate pybind11 includes

Simplify header files by removing redundant pybind11 includes, reducing clutter and improving maintainability.

* docs: add ROS2 message support details and example links

Enhance the documentation by including information on ROS2 message formats and how to generate corresponding Python code. This provides users with clearer guidance on integrating ROS2 functionality within the project.

* chore: update copyright information to reflect new authorship

Update copyright year and licensing information across multiple configuration files and application scripts to maintain proper attribution for the AimRT project.

* feat: add ROS2 RPC client and server examples with configurations

Introduce example applications for ROS2 RPC communication, including client and server scripts, along with necessary configuration files. Implement a build script to set up the protocol buffers correctly, enhancing usability by supporting both HTTP and gRPC backends.

* feat: update RPC request and response handling

Refactor the RPC client and server implementations to utilize the updated request and response types from the example ROS2 service, improving code clarity and ensuring compatibility with the new service structure.

* refactor: simplify RPC response handling

Remove unnecessary response object instantiation and streamline the response creation process for improved clarity and efficiency in the RPC invocation logic.

* refactor: streamline publisher and subscriber method names

Simplify method names related to publishing and subscribing to enhance readability and maintainability. Transition from a prefix-based naming convention to a more concise approach, ensuring consistency across the codebase.

* chore: update configuration file paths for gRPC client and server examples

Align the configuration file paths in the example scripts to use the gRPC-specific settings, ensuring the correct configuration is utilized for both the client and server.

* refactor: rename internal methods for clarity

Improve naming conventions for protobuf message type functions, enhancing code readability and maintainability. Update related usages to keep consistent across the runtime and RPC generator.

* feat: add service type validation and improve bash scripts

Introduce a new bash script for debugging the ROS2 client and enhance existing scripts with proper sourcing. Implement service type validation to ensure the reliability of service registration. This improves the robustness of RPC interactions and simplifies the setup process for developers.

* feat: extend RPC response data structure and improve logging

Add additional data types to the RosTestRpc response for enhanced functionality. Set logging level to trace for more detailed output during execution.

* feat: add support for static and dynamic arrays in RPC service

Enhance the RPC service by introducing static and dynamic array types for various data types, enabling more complex data handling in requests and responses. This improves flexibility and usability for clients interacting with the service.

* feat: add support for new ROS message types

Enhance the RPC server to handle additional ROS message types, including dynamic and static arrays of custom data structures. This improves the server's data handling capabilities and aligns with the updated service specifications.

* refactor: improve type mapping and message copy functions

Update type mappings for better consistency and clarity. Enhance the readability of message copying functions by formatting multi-line expressions.

* refactor: improve message copying and moving functions

Enhance the handling of ROS message copying and moving by updating the parameter order and adding tailored move functions for better memory management. This optimizes the performance of message manipulations and ensures cleaner, more maintainable code.

* refactor: simplify message type support functions

Remove inline implementations and enhance readability of message type support functions, while maintaining functionality and improving code organization.

* refactor: streamline ROS message handling and improve introspection support

Enhance readability and maintainability by cleaning up function definitions related to ROS message type support. Implement a more robust method for obtaining message members information, addressing potential failure scenarios. Adjust example service implementation for consistency in dynamic data handling.

* fix: ensure pointers for ROS message creation and destruction are valid

Throw runtime exceptions if the function pointers for creating or destroying ROS messages are null, enhancing error handling and stability during object initialization.

* feat: enhance RosTestRpc method signature for better type safety

Improve the RosTestRpc method by adding overloads for different argument configurations, ensuring clearer type expectations and reducing potential runtime errors. This change aids in maintaining robust interactions with the ROS2 service framework.

* refactor: simplify parameter naming in RPC proxy function

Rename the `ctx_ref` parameter to `ctx` in the RPC function signature for cleaner and more concise code. This enhances readability and maintains consistency across the codebase.

* fix: restrict static array sizes in RosTestRpc service

Limit the size of static array fields to three elements each for better resource management and to prevent potential overflow issues.

* feat: enhance python runtime generation for ROS2 services

Add support for generating code from ROS2 service definitions, improving interoperability with the aimrt framework. Update build scripts to include new generation scripts and ensure file dependencies are configured correctly.

* refactor: update RPC service and client implementations to use ROS2 naming convention

Modify scripts to utilize the new aimrt_py-gen-ros2-rpc tool and update service and client references accordingly for consistency and clarity. Remove deprecated code to streamline functionality and improve maintainability.

* chore: update .gitignore to exclude generated files

Add pattern to ignore *_aimrt_rpc_ros2.py files to keep the repository clean from auto-generated files.

* refactor: streamline response structure for RosTestRpc

Remove unused fields from RosTestRpc response to simplify the data structure and enhance clarity. Adjust logging level for better control over output verbosity.

* chore: update log level to INFO for consistency

Adjust the core log level to INFO to ensure a more standard logging output and reduce unnecessary verbosity in the logs.

* docs: add README for ros2 rpc examples

Provide detailed instructions and explanations for setting up and running Python RPC examples using ros2 protocols with HTTP, gRPC, and native ros2 backends.

* feat: enhance aimrt_py channel with rpc support for ros2 message types

Add support for ros2 message types in both aimrt_py channel and rpc to improve compatibility and functionality.

* docs: add ros2_rpc example to Python interface section

Include a new example link to enhance the documentation and provide more resources for users exploring the Python interfaces.

* chore: add spacing for improved readability in service proxy class

Improve code readability by adding blank lines in the service proxy class definition.

* chore: standardize argument names for RPC generator

Update argument names in the RPC generation script and implementation for consistency, enhancing readability and usability.

* docs: update RPC documentation to include ROS2 Srv support

Add examples and instructions for using ROS2 Srv in RPC, enhancing integration options for developers. Adjust existing protobuf references for clarity and consistency.

* style: standardize whitespace in RosTestRpc.srv

Improve readability by ensuring consistent spacing in the service definition.

* build: exclude ROS2 specific files from the build if not using ROS2

Remove ROS2-related source files from the compilation when the project is configured without ROS2 support, helping to streamline the build process and avoid unnecessary dependencies.

* style: add missing newline at end of file for consistency

* refactor: rename copy function to improve clarity

Refactor the function used for copying messages within dynamic arrays to better convey its functionality. This change enhances code readability and maintainability.

---------

Co-authored-by: zhangyi <zhangyi@agibot.com>
2024-11-21 11:48:44 +08:00

13 KiB
Raw Blame History

Rpc

相关链接

参考示例:

协议

协议用于确定 RPC 中客户端和服务端的消息格式。一般来说,协议都是使用一种与具体的编程语言无关的 IDL ( Interface description language )描述,然后由某种工具转换为各个语言的代码。对于 RPC 来说,这里需要两个步骤:

  • 参考Channel章节中的介绍,开发者需要先利用一些官方的工具为协议文件中的消息类型生成指定编程语言中的代码;
  • 开发者需要使用 AimRT 提供的工具,为协议文件中服务定义生成指定编程语言中的代码;

Protobuf

Protobuf是一种由 Google 开发的、用于序列化结构化数据的轻量级、高效的数据交换格式,是一种广泛使用的 IDL。它不仅能够描述消息结构还提供了service语句来定义 RPC 服务。

在使用时,开发者需要先定义一个.proto文件,在其中定义消息结构和 RPC 服务。例如rpc.proto

syntax = "proto3";

package example;

message ExampleReq {
  string msg = 1;
  int32 num = 2;
}

message ExampleRsp {
  uint64 code = 1;
  string msg = 2;
}

service ExampleService {
  rpc ExampleFunc(ExampleReq) returns (ExampleRsp);
}

然后使用 Protobuf 官方提供的 protoc 工具进行转换,生成消息结构部分的 Python 代码,例如:

protoc --python_out=. rpc.proto

这将生成rpc_pb2.py文件,包含了根据定义的消息类型生成的 Python 接口。

在这之后,还需要使用 AimRT 提供的 protoc 插件,生成服务定义部分的 Python 桩代码,例如:

protoc --aimrt_rpc_out=. --plugin=protoc-gen-aimrt_rpc=./protoc_plugin_py_gen_aimrt_py_rpc.py rpc.proto

这将生成rpc_aimrt_rpc_pb2.py文件,包含了根据定义的服务生成的 Python 接口,我们的业务代码中需要 import 此文件。

ROS2 Srv

ROS2 Srv 是一种用于在 ROS2 中进行 RPC 定义的格式。在使用时,开发者需要先定义一个 ROS2 Package在其中定义一个.srv文件,比如example.srv

byte[]  data
---
int64   code

其中,以---来分割 Req 和 Rsp 的定义。

aimrt_py 安装时会自动安装一个命令行工具 aimrt_py-gen-ros2-rpc,用于根据 ROS2 Srv 文件生成 AimRT Python 的 RPC 桩代码。

aimrt_py-gen-ros2-rpc --pkg_name=example_pkg --srv_file=./example.srv --output_path=./

其中 pkg_name 是 ROS2 Package 的名称srv_file 是 ROS2 Srv 文件的路径output_path 是生成的桩代码的输出路径。这将生成一个example_aimrt_rpc_ros2.py文件,包含了根据定义的服务生成的 Python 接口,我们的业务代码中需要 import 此文件。

需要注意的是,aimrt_py-gen-ros2-rpc 只会生成 Req 和 Rsp 的定义,不会生成消息结构部分的代码,消息结构部分代码仍然需要使用者自行生成(即构建 ROS2 Package 并生成消息结构代码, 详情可以参考 ROS2 官方文档)。

RpcHandle

模块可以通过调用CoreRef句柄的GetRpcHandle()接口,获取RpcHandleRef句柄。一般情况下,开发者不会直接使用RpcHandleRef直接提供的接口,而是根据 RPC IDL 文件生成一些桩代码,对RpcHandleRef句柄做一些封装,然后在业务代码中使用这些经过封装的接口。

这些经过封装的接口的具体形式将在本文档后续章节介绍。开发者在使用 RPC 功能时需要按照以下步骤使用这些接口:

  • Client 端:
    • Initialize阶段,调用注册 RPC Client 方法的接口;
    • Start阶段,调用 RPC Invoke 的接口,以实现 RPC 调用;
  • Server 端:
    • Initialize阶段,注册 RPC Server 服务的接口;

RpcStatus

在 RPC 调用或者 RPC 处理时,使用者可以通过一个RpcStatus类型的变量获取 RPC 过程中的错误情况,其包含的接口如下:

  • OK()->bool :是否成功;
  • Code()->int :错误码;
  • ToString()->str :转字符串;

RpcStatus类型非常轻量,其中只包含一个错误码字段。使用者可以通过构造函数或 Set 方法设置这个 Code可以通过 Get 方法获取这个 Code。错误码的枚举值可以参考{{ 'rpc_status_base.h'.format(code_site_root_path_url) }}文件中的定义。

请注意,RpcStatus中的错误信息一般仅表示框架层面的错误,例如服务未找到、网络错误或者序列化错误等,供开发者排查框架层面的问题。如果开发者需要返回业务层面的错误,建议在业务包中添加相应的字段。

RpcContext

RpcContext 是 RPC 调用时上下文信息,开发者可以在 RPC 调用时设置一些上下文信息例如超时时间、Meta 信息等,具体接口如下:

  • CheckUsed()->bool :检查 Context 是否被使用;
  • SetUsed()->None :设置 Context 为已使用;
  • Reset()->None :重置 Context
  • GetType()->aimrt_rpc_context_type_t :获取 Context 类型;
  • Timeout()->datetime.timedelta :获取超时时间;
  • SetTimeout(timeout: datetime.timedelta)->None :设置超时时间;
  • SetMetaValue(key: str, value: str)->None :设置元数据;
  • GetMetaValue(key: str)->str :获取元数据;
  • GetMetaKeys()->List[str] :获取所有元数据键值对中的键列表;
  • SetToAddr(addr: str)->None :设置目标地址;
  • GetToAddr()->str :获取目标地址;
  • SetSerializationType(serialization_type: str)->None :设置序列化类型;
  • GetSerializationType()->str :获取序列化类型;
  • GetFunctionName()->str :获取函数名称;
  • SetFunctionName(func_name: str)->None :设置函数名称;
  • ToString()->str :获取上下文信息,以字符串形式返回可读性高的信息;

RpcContextRefRpcContext的引用类型,除不具备Reset接口外,其他接口与RpcContext完全相同。

aimrt_rpc_context_type_t 是一个枚举类型,定义了上下文类型,具体值为AIMRT_RPC_CLIENT_CONTEXTAIMRT_RPC_SERVER_CONTEXT,表明这是客户端还是服务端的上下文。

Client

在 AimRT Python RPC 桩代码工具生成的代码里,如xxx_aimrt_rpc_pb2.py文件里,提供了XXXProxy类型,开发者基于此 Proxy 接口来发起 RPC 调用。此接口是同步型接口,使用此 Proxy 接口发起 RPC 调用后会阻塞当前线程,直到收到回包或请求超时。

使用该 Proxy 发起 RPC 调用非常简单,一般分为以下几个步骤:

  • Step 0:引用根据 protobuf 协议生成的桩代码包,例如xxx_aimrt_rpc_pb2.py
  • Step 1:在Initialize阶段调用该 Proxy 的RegisterClientFunc静态方法注册 RPC Client
  • Step 2:在Start阶段里某个业务函数里发起 RPC 调用:
    • Step 2-1:创建一个 Proxy 实例,构造参数是RpcHandleRef
    • Step 2-2:创建 Req并填充 Req 内容;
    • Step 2-3:【可选】创建 ctx设置超时等信息
    • Step 2-4:基于 proxy传入 ctx、Req发起 RPC 调用,同步等待 RPC 调用结束,保证在整个调用周期里 ctx、Req 都保持有效且不会改动,最终获取返回的 status 和 Rsp
    • Step 2-5:解析 status 和 Rsp

以下是一个使用 AimRT Python 基于 protobuf 协议进行 RPC Client 调用的示例,通过 Create Module 方式拿到CoreRef句柄。如果是基于Module模式在Initialize方法中拿到CoreRef句柄,使用方式也类似:

import aimrt_py
import threading
import time
import datetime

from google.protobuf.json_format import MessageToJson
import rpc_pb2
import rpc_aimrt_rpc_pb2

def main():
    aimrt_core = aimrt_py.Core()

    # Initialize
    core_options = aimrt_py.CoreOptions()
    core_options.cfg_file_path = "path/to/cfg/xxx_cfg.yaml"
    aimrt_core.Initialize(core_options)

    # Create Module
    module_handle = aimrt_core.CreateModule("NormalRpcClientPyModule")

    # Register rpc client
    rpc_handle = module_handle.GetRpcHandle()
    ret = rpc_aimrt_rpc_pb2.ExampleServiceProxy.RegisterClientFunc(rpc_handle)
    assert ret, "Register client failed."

    # Start
    thread = threading.Thread(target=aimrt_core.Start)
    thread.start()

    # Sleep for seconds
    time.sleep(1)

    # Call rpc
    proxy = rpc_aimrt_rpc_pb2.ExampleServiceProxy(rpc_handle)

    req = rpc_pb2.GetFooDataReq()
    req.msg = "example msg"

    ctx = aimrt_py.RpcContext()
    ctx.SetTimeout(datetime.timedelta(seconds=30))
    ctx.SetMetaValue("key1", "value1")
    status, rsp = proxy.GetFooData(ctx, req)

    aimrt_py.info(module_handle.GetLogger(),
                  f"Call rpc done, "
                  f"status: {status.ToString()}, "
                  f"req: {MessageToJson(req)}, "
                  f"rsp: {MessageToJson(rsp)}")

    # Shutdown
    aimrt_core.Shutdown()

    thread.join()

if __name__ == '__main__':
    main()

Server

在 AimRT Python RPC 桩代码工具生成的代码里,如xxx_aimrt_rpc_pb2.py文件里,提供了一个继承了aimrt_py.ServiceBase的 Service 基类,开发者需要继承该 Service 基类并实现其中的虚接口。此 Service 接口是同步型接口,开发者只能在 handle 中阻塞的完成所有操作并在最后返回回包。

使用该接口提供 RPC 服务,一般分为以下几个步骤:

  • Step 0:引用根据 protobuf 协议生成的桩代码包,例如xxx_aimrt_rpc_pb2.py
  • Step 1:开发者实现一个 Impl 类,继承包中的XXXService,并实现其中的虚接口,接口形式为(ctx, req)->status, rsp
    • Step 1-1:解析 Ctx 和 Req并填充 Rsp
    • Step 1-2:返回RpcStatus和 Rsp
  • Step 2:在Initialize阶段调用RpcHandleRefRegisterService方法注册 RPC Service

以下是一个使用 AimRT Python 基于 protobuf 协议进行 RPC Service 处理的示例,通过 Create Module 方式拿到CoreRef句柄。如果是基于Module模式在Initialize方法中拿到CoreRef句柄,使用方式也类似:

import aimrt_py
import threading
import signal

from google.protobuf.json_format import MessageToJson
import rpc_pb2
import rpc_aimrt_rpc_pb2


global_aimrt_core = None


def signal_handler(sig, frame):
    global global_aimrt_core

    if (global_aimrt_core and (sig == signal.SIGINT or sig == signal.SIGTERM)):
        global_aimrt_core.Shutdown()
        return

    sys.exit(0)


class ExampleServiceImpl(rpc_aimrt_rpc_pb2.ExampleService):
    def __init__(self, logger):
        super().__init__()
        self.logger = logger

    @staticmethod
    def PrintMetaInfo(logger, ctx_ref):
        meta_keys = ctx_ref.GetMetaKeys()
        for key in meta_keys:
            aimrt_py.info(logger, f"meta key: {key}, value: {ctx_ref.GetMetaValue(key)}")

    def GetFooData(self, ctx_ref, req):
        rsp = rpc_pb2.GetFooDataRsp()
        rsp.msg = "echo " + req.msg

        ExampleServiceImpl.PrintMetaInfo(self.logger, ctx_ref)
        aimrt_py.info(self.logger,
                      f"Server handle new rpc call. "
                      f"context: {ctx_ref.ToString()}, "
                      f"req: {MessageToJson(req)}, "
                      f"return rsp: {MessageToJson(rsp)}")

        return aimrt_py.RpcStatus(), rsp

def main():
    signal.signal(signal.SIGINT, signal_handler)
    signal.signal(signal.SIGTERM, signal_handler)

    aimrt_core = aimrt_py.Core()

    global global_aimrt_core
    global_aimrt_core = aimrt_core

    # Initialize
    core_options = aimrt_py.CoreOptions()
    core_options.cfg_file_path = "path/to/cfg/xxx_cfg.yaml"
    aimrt_core.Initialize(core_options)

    # Create Module
    module_handle = aimrt_core.CreateModule("NormalRpcServerPymodule")

    # Register rpc service
    service = ExampleServiceImpl(module_handle.GetLogger())
    ret = module_handle.GetRpcHandle().RegisterService(service)
    assert ret, "Register service failed."

    # Start
    thread = threading.Thread(target=aimrt_core.Start)
    thread.start()

    while thread.is_alive():
        thread.join(1.0)

if __name__ == '__main__':
    main()

基于 ROS2 Srv 协议的 RPC 调用和处理,除数据类型不同外,其他使用方式与基于 protobuf 协议的 RPC 调用和处理基本一致。

完整示例请参考: