Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/audio/module_adapter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,11 @@ sequenceDiagram
deactivate DP
```

A sinkless DP module derives its period from the first source's
`min_available` value (its IBS) and audio format. This schedules input-only
detectors according to data arrival; a zero or otherwise invalid period is
rejected by the normal minimum-period check during prepare.

## Error Handling and Memory Sandboxing

* **Sandboxing (`mod_balloc_align`, `z_impl_mod_fast_get`, `z_impl_mod_free`)**: Since third-party DSP code is treated as semi-untrusted in memory lifetimes, module allocations grab slices from a dedicated component `dp_heap_user` heap instead of the global system heap (`mod_heap_info`). The wrapper automatically prunes leaked objects (`mod_free_all(mod)`) during teardown by keeping an `objpool` of all resource containers.
Expand Down
21 changes: 18 additions & 3 deletions src/audio/module_adapter/iadk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,19 @@ classDiagram

Because the actual module resides in an external binary, it requires a "System Agent" to correctly instantiate the C++ objects during the component's `init` phase.

The adapter accepts IADK API 4.5.0 and 4.5.2 binaries. API 4.5.2
extends the detector interface with `WritePattern()`, adds
`SystemAgentInterface3::GetBssBase()`, and appends a versioned lookup
entry to the system service table. The existing table and vtable slots
keep their API 4.5.0 layout.

1. The OS host driver sends an IPC `INIT_INSTANCE` command for the module.
2. The `system_agent_start()` function intercepts this, invokes the dynamic module's `create_instance` entry point (which invokes a `ModuleFactory`).
3. The `SystemAgent` deduces the pin count (interfaces) and initial pipeline configurations using `ModuleInitialSettingsConcrete`.
4. The factory allocates the concrete algorithm and checks it back into SOF through `SystemAgent::CheckIn`.
3. If the entry point provides a null module placeholder, the `SystemAgent`
resolves it to the instance BSS allocated by the library manager after
verifying that the module object fits in that region.
4. The `SystemAgent` deduces the pin count (interfaces) and initial pipeline configurations using `ModuleInitialSettingsConcrete`.
5. The factory allocates the concrete algorithm and checks it back into SOF through `SystemAgent::CheckIn`.

```mermaid
sequenceDiagram
Expand All @@ -64,6 +73,7 @@ sequenceDiagram

IPC->>SA: Trigger Mod Creation
SA->>Fac: CI invokes create_instance
SA->>SA: Resolve instance BSS placeholder
Fac->>Fac: Deduce BaseModuleCfgExt
Fac->>Mod: operator new instantiate

Expand All @@ -79,7 +89,12 @@ A significant task of `IadkModuleAdapter_Process` is converting SOF's underlying

Instead of letting the module directly touch the SOF `comp_buffer` (which could change with SOF version updates), the adapter uses the abstraction APIs (`source_get_data` / `sink_get_buffer`) and wraps them:

1. Request raw continuous memory pointers from `source_get_data()`.
1. Limit each input to one source `min_available` portion (one IBS), then
request its continuous memory pointer from `source_get_data()`.
2. Construct an `intel_adsp::InputStreamBuffer` pointing to that continuous memory chunk.
3. Call the IADK `processing_module_.Process()`.
4. Release precisely the amount of consumed data using `source_release_data()`.

Input-only detector endpoints are processed even when no audio sink is
connected. Their output descriptors remain empty, so the module can consume
and inspect each IBS without producing downstream audio.
8 changes: 6 additions & 2 deletions src/audio/module_adapter/iadk/iadk_module_adapter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ int IadkModuleAdapter::IadkModuleAdapter_Process(struct sof_source **sources,
{
int ret = 0;

if ((num_of_sources > 0) && (num_of_sinks > 0)) {
if (num_of_sources > 0) {
intel_adsp::InputStreamBuffer input_stream_buffers[INPUT_PIN_COUNT];
intel_adsp::OutputStreamBuffer output_stream_buffers[OUTPUT_PIN_COUNT];
for (int i = 0; i < (int)num_of_sources; i++) {
Expand All @@ -55,6 +55,11 @@ int IadkModuleAdapter::IadkModuleAdapter_Process(struct sof_source **sources,

intel_adsp::InputStreamFlags flags = {};
i_size = source_get_data_available(sources[i]);
size_t min_available = source_get_min_available(sources[i]);

if (min_available && i_size > min_available)
i_size = min_available;

ret = source_get_data(sources[i], i_size, (const void **)&input,
(const void **)&input_start, &input_end);
if (ret != 0)
Expand Down Expand Up @@ -255,4 +260,3 @@ void* operator new(size_t size, intel_adsp::OutputStreamBuffer* placeholder) thr
return placeholder;
}
#endif

64 changes: 61 additions & 3 deletions src/audio/module_adapter/iadk/system_agent.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,15 @@
#include <sof/audio/module_adapter/library/native_system_service.h>
#include <sof/audio/module_adapter/library/native_system_agent.h>

/* sof/lib_manager.h pulls C-only component headers into this C++ unit. */
struct sof_man_module;
extern "C" const struct sof_man_module *
lib_manager_get_module_manifest(const uint32_t module_id);
extern "C" void
lib_manager_get_instance_bss_address(uint32_t instance_id,
const struct sof_man_module *mod,
void __sparse_cache **va_addr, size_t *size);

using namespace intel_adsp;
using namespace intel_adsp::system;
using namespace dsp_fw;
Expand All @@ -38,6 +47,21 @@ namespace intel_adsp
namespace system
{

static void *system_agent_get_instance_bss(uint32_t module_id,
uint32_t instance_id,
size_t *size)
{
const struct sof_man_module *mod = lib_manager_get_module_manifest(module_id);
void __sparse_cache *base = NULL;

*size = 0;
if (!mod)
return NULL;

lib_manager_get_instance_bss_address(instance_id, mod, &base, size);
return reinterpret_cast<void *>(base);
}

/* Structure storing handles to system service operations */
const APP_TASK_DATA AdspSystemService SystemAgent::system_service_ = {
native_system_service_log_message,
Expand All @@ -47,6 +71,7 @@ const APP_TASK_DATA AdspSystemService SystemAgent::system_service_ = {
native_system_service_create_notification,
native_system_service_send_notif_msg,
native_system_service_get_interface,
native_system_service_get_interface_versioned,
};

SystemAgent::SystemAgent(uint32_t module_id,
Expand Down Expand Up @@ -79,6 +104,21 @@ void SystemAgent::CheckIn(ProcessingModuleInterface& processing_module,
log_handle = reinterpret_cast<LogHandle*>(log_handle_);
}

void SystemAgent::CheckInDetector(DetectorModuleInterface& processing_module,
ModuleHandle &module_handle,
LogHandle *&log_handle)
{
CheckIn(static_cast<ProcessingModuleInterface&>(processing_module),
module_handle, log_handle);
}

void *SystemAgent::GetBssBase(void)
{
size_t size = 0;

return system_agent_get_instance_bss(module_id_, instance_id_, &size);
}

int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
ModulePlaceholder *module_placeholder,
size_t processing_module_size,
Expand All @@ -87,6 +127,19 @@ int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
void *obfuscated_parent_ppl,
void **obfuscated_modinst_p)
{
if (!module_placeholder) {
size_t bss_size;
void *bss_base = system_agent_get_instance_bss(module_id_, instance_id_,
&bss_size);

if (!bss_base || processing_module_size > bss_size)
return -ENOMEM;

module_placeholder = reinterpret_cast<ModulePlaceholder *>(bss_base);
}

module_size_ = processing_module_size;

IoPinsInfo pins_info;
const dsp_fw::DwordArray& cfg_ipc_msg =
*reinterpret_cast<const dsp_fw::DwordArray*>(obfuscated_mod_cfg);
Expand All @@ -110,10 +163,16 @@ int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
settings.DeduceBaseModuleCfgExt(prerequisites.input_pins_count,
prerequisites.output_pins_count);

module_factory.Create(*this, module_placeholder, ModuleInitialSettings(settings), pins_info);
int ret = module_factory.Create(*this, module_placeholder,
ModuleInitialSettings(settings), pins_info);
if (ret)
return ret;

if (!module_handle_)
return -EINVAL;

IadkModuleAdapter& module_adapter = *reinterpret_cast<IadkModuleAdapter*>(module_handle_);
*obfuscated_modinst_p = &module_adapter;
reinterpret_cast<intel_adsp::ProcessingModuleInterface*>(module_placeholder)->Init();
return 0;
}

Expand Down Expand Up @@ -148,4 +207,3 @@ extern "C" void __cxa_pure_virtual() __attribute__((weak));
void __cxa_pure_virtual()
{
}

18 changes: 15 additions & 3 deletions src/audio/module_adapter/library/native_system_service.c
Original file line number Diff line number Diff line change
Expand Up @@ -158,9 +158,20 @@ AdspErrorCode native_system_service_send_notif_msg(enum notification_target noti
AdspErrorCode native_system_service_get_interface(enum interface_id id,
struct system_service_iface **iface)
{
if (id < 0)
if (id < 0 || !iface)
return ADSP_INVALID_PARAMETERS;
return ADSP_NO_ERROR;

*iface = NULL;
return ADSP_INVALID_PARAMETERS;
Comment on lines +161 to +165
}

AdspErrorCode native_system_service_get_interface_versioned(enum interface_id id,
uint32_t version,
struct system_service_iface **iface)
{
(void)version;

return native_system_service_get_interface(id, iface);
}

const APP_TASK_DATA struct native_system_service native_system_service = {
Expand All @@ -171,6 +182,7 @@ const APP_TASK_DATA struct native_system_service native_system_service = {
.vec_memset = native_system_service_vec_memset,
.notification_create = native_system_service_create_notification,
.notification_send = native_system_service_send_notif_msg,
.get_interface = native_system_service_get_interface
.get_interface = native_system_service_get_interface,
.get_interface_versioned = native_system_service_get_interface_versioned
}
};
22 changes: 20 additions & 2 deletions src/audio/module_adapter/module_adapter.c
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,18 @@ static void module_adapter_calculate_dp_period(struct comp_dev *dev)

}

/* Sinkless modules derive their deadline from one input buffer. */
if (period == UINT32_MAX && mod->num_of_sources > 0) {
size_t frame_bytes = source_get_frame_bytes(mod->sources[0]);
unsigned int rate = source_get_rate(mod->sources[0]);

if (frame_bytes && rate)
period = 1000000ULL * source_get_min_available(mod->sources[0]) /
(frame_bytes * rate);
else
period = 0;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previously in this case dev->period would be set to UINT32_MAX. Is this change deliberate? What's its purpose? This seems to be an error case anyway, right?

}

dev->period = period;
}
#endif /* CONFIG_ZEPHYR_DP_SCHEDULER */
Expand Down Expand Up @@ -473,12 +485,18 @@ int module_adapter_prepare(struct comp_dev *dev)
* Hence check for NULL.
*/
sink = comp_dev_get_first_data_consumer(dev);
if (!sink) {
if (!sink && !IS_PROCESSING_MODE_SINK_SOURCE(mod)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are sinkless modules only allowed in DP mode or in LL too?

comp_err(dev, "no sink present on period size calculation");
return -EINVAL;
}

mod->period_bytes = audio_stream_period_bytes(&sink->stream, dev->frames);
/* A sinkless module still needs at least one source. */
if (!sink && !comp_dev_get_first_data_producer(dev)) {
comp_err(dev, "no source or sink buffer connected");
return -EINVAL;
}

mod->period_bytes = sink ? audio_stream_period_bytes(&sink->stream, dev->frames) : 0;
comp_dbg(dev, "got period_bytes = %u", mod->period_bytes);

/* no more to do for sink/source mode */
Expand Down
6 changes: 5 additions & 1 deletion src/include/module/module/system_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ enum interface_id {
INTERFACE_ID_SDCA = 0x1002, /* See SdcaInterface */
INTERFACE_ID_ASYNC_MESSAGE_SERVICE = 0x1003, /* See AsyncMessageInterface */
INTERFACE_ID_AM_SERVICE = 0x1005, /* Reserved for ADSP system */
INTERFACE_ID_KPB_SERVICE = 0x1006 /* See KpbInterface */
INTERFACE_ID_KPB_SERVICE = 0x1006, /* See KpbInterface */
INTERFACE_ID_TIMESTAMPING_SERVICE = 0x1007 /* See TimestampingInterface */
};

/* sub interface definition.
Expand Down Expand Up @@ -109,5 +110,8 @@ struct system_service {
uint32_t actual_payload_size);

AdspErrorCode (*get_interface)(enum interface_id id, struct system_service_iface **iface);

AdspErrorCode (*get_interface_versioned)(enum interface_id id, uint32_t version,
struct system_service_iface **iface);
};
#endif /* MODULE_MODULE_SYSTEM_SERVICE_H */
5 changes: 4 additions & 1 deletion src/include/sof/audio/module_adapter/iadk/api_version.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,14 @@

#define IADK_MODULE_API_MAJOR_VERSION 4
#define IADK_MODULE_API_MIDDLE_VERSION 5
#define IADK_MODULE_API_MINOR_VERSION 0
#define IADK_MODULE_API_MINOR_VERSION 2

#define IADK_MODULE_API_CURRENT_VERSION MODULE_API_VERSION_ENCODE(IADK_MODULE_API_MAJOR_VERSION, \
IADK_MODULE_API_MIDDLE_VERSION, IADK_MODULE_API_MINOR_VERSION)

/* API 4.5.0 remains compatible with the extended interface tables. */
#define IADK_MODULE_API_VERSION_4_5_0 MODULE_API_VERSION_ENCODE(4, 5, 0)

/* Defines the size of the space reserved within ModuleHandle for SOF to store its private data.
* This size comes from the IADK header files and must match the IADK API version.
* Please do not modify this value!
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,10 @@ namespace intel_adsp
virtual void OnStreamState(uint64_t counter,
uint32_t stream_index,
State state) = 0;

/*! Write a pattern when an output stream resumes processing. */
virtual void WritePattern(uint16_t stream_index,
OutputStreamBuffer *output_buffer) = 0;
};
} /*namespace intel_adsp */

Expand Down
11 changes: 10 additions & 1 deletion src/include/sof/audio/module_adapter/iadk/system_agent.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ namespace system
* A SystemAgent can only be delivered by the ADSP System.
* Once registered, a ModuleHandle instance can be handled by the ADSP System.
*/
class SystemAgent : public intel_adsp::SystemAgentInterface
class SystemAgent : public intel_adsp::SystemAgentInterface3
{
public:
SystemAgent(uint32_t module_id,
Expand All @@ -34,6 +34,15 @@ namespace system
intel_adsp::ModuleHandle & module_handle,
intel_adsp::LogHandle * &log_handle) /*override*/;

/*! \brief Registers a detector module. */
virtual void CheckInDetector(
DetectorModuleInterface & processing_module,
ModuleHandle & module_handle,
LogHandle * &log_handle) /*override*/;

/*! \brief Returns the BSS base for the current module instance. */
virtual void *GetBssBase(void) /*override*/;

/*! \return a value part of error code list defined within the adsp_error.h*/
virtual int CheckIn(intel_adsp::ProcessingModuleFactoryInterface & module_factory,
intel_adsp::ModulePlaceholder * module_placeholder,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,14 @@ namespace internal
) = 0;
};

class SystemAgentInterface3 : public SystemAgentInterface2
{
public:

/*! \brief Gets the BSS base for the current module instance. */
virtual void *GetBssBase(void) = 0;
};

} /* namespace intel_adsp */

#endif /* _ADSP_SYSTEM_AGENT_H_ */
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,18 @@ AdspErrorCode native_system_service_send_notif_msg(enum notification_target noti
AdspErrorCode native_system_service_get_interface(enum interface_id id,
struct system_service_iface **iface);

/**
* \brief Retrieve a versioned system service interface.
*
* \param id Service interface identifier.
* \param version Requested interface version.
* \param iface Location for the returned interface pointer.
* \return ADSP_INVALID_PARAMETERS if the interface is unavailable.
*/
AdspErrorCode native_system_service_get_interface_versioned(enum interface_id id,
uint32_t version,
struct system_service_iface **iface);

extern const struct native_system_service native_system_service;

#ifdef __cplusplus
Expand Down
3 changes: 2 additions & 1 deletion src/library_manager/lib_manager.c
Original file line number Diff line number Diff line change
Expand Up @@ -628,7 +628,8 @@ static enum buildinfo_mod_type lib_manager_get_module_type(const struct sof_man_
/* Check if module is IADK */
if (IS_ENABLED(CONFIG_INTEL_MODULES) &&
build_info->format == IADK_MODULE_API_BUILD_INFO_FORMAT &&
build_info->api_version_number.full == IADK_MODULE_API_CURRENT_VERSION) {
(build_info->api_version_number.full == IADK_MODULE_API_CURRENT_VERSION ||
build_info->api_version_number.full == IADK_MODULE_API_VERSION_4_5_0)) {
return MOD_TYPE_IADK;
} else {
/* Check if module is NOT native */
Expand Down
Loading