Wingman AI Version
2.1.1
Bug Description
On the old version of MSFS skill there were a lot of tokens added into the prompt to explain to the wingman all the available commands for the MSFS SDK which are not at at all common sense or in a predictable format, but this burned too many tokens.
The revised version of the skill cuts a lot of tokens but now leaves too much for the AI to guess about in calling commands in the SDK which a good portion of the time leads to the wingman saying they performed a function in game (and trying to call the skill) but coming up empty because the command was not in the exact format expected by the SDK, which doesnt throw an error it just fails silently.
The latest version of the skill attempts to solve that in part by implementing a tool where the wingman can search for the format of likely commands and then pick the best option. But in practice more often than not the pattern is asking the wingman to do something, wingman guesses and fails, and then user has to say “that didn’t work search for another command.”
This might be solved in various ways including maybe enhanced prompting for the skill, encouraging wingman to look up commands to match the user’s intent before responding, maybe implementing specific tools for the most common commands like throttle settings, parking brake, flaps, toggling auto pilot, etc., or perhaps seeing if some sort of “error” or feedback can be sent to the llm that the command was not in an expected for,at and it should try searching for the proper command as part of the tool response. Not sure.
Steps to Reproduce
Use MSFS skill to try to control plane for extended session with various commands
Wingman Configuration (Wingman.yaml)
Settings (settings.yaml)
Log File
Screenshots / Error Messages
N/A
Additional Context
No response
Secrets Check
Wingman AI Version
2.1.1
Bug Description
On the old version of MSFS skill there were a lot of tokens added into the prompt to explain to the wingman all the available commands for the MSFS SDK which are not at at all common sense or in a predictable format, but this burned too many tokens.
The revised version of the skill cuts a lot of tokens but now leaves too much for the AI to guess about in calling commands in the SDK which a good portion of the time leads to the wingman saying they performed a function in game (and trying to call the skill) but coming up empty because the command was not in the exact format expected by the SDK, which doesnt throw an error it just fails silently.
The latest version of the skill attempts to solve that in part by implementing a tool where the wingman can search for the format of likely commands and then pick the best option. But in practice more often than not the pattern is asking the wingman to do something, wingman guesses and fails, and then user has to say “that didn’t work search for another command.”
This might be solved in various ways including maybe enhanced prompting for the skill, encouraging wingman to look up commands to match the user’s intent before responding, maybe implementing specific tools for the most common commands like throttle settings, parking brake, flaps, toggling auto pilot, etc., or perhaps seeing if some sort of “error” or feedback can be sent to the llm that the command was not in an expected for,at and it should try searching for the proper command as part of the tool response. Not sure.
Steps to Reproduce
Use MSFS skill to try to control plane for extended session with various commands
Wingman Configuration (Wingman.yaml)
N/ASettings (settings.yaml)
N/ALog File
Screenshots / Error Messages
N/A
Additional Context
No response
Secrets Check
secrets.yamlor any API keys in this report.