Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitter Spaces Voice Client #301

Open
madjin opened this issue Nov 13, 2024 · 1 comment
Open

Twitter Spaces Voice Client #301

madjin opened this issue Nov 13, 2024 · 1 comment
Labels
Bounty Use this to assign a bounty to an issue enhancement New feature or request

Comments

@madjin
Copy link
Collaborator

madjin commented Nov 13, 2024

Twitter Spaces Voice Client

Parity with Discord Voice Client, but on Twitter Space

  • Must listen to and respond with voice
  • Must be able to join a twitter space from a linked tweet
  • Must be able to accept speaker role

Reward: $1000 USD in $ai16z + $1000 USD in $degenai as seen on https://ai16z.github.io/

@madjin madjin added enhancement New feature or request Bounty Use this to assign a bounty to an issue labels Nov 13, 2024
@pukadev
Copy link

pukadev commented Nov 14, 2024

def use the openai real time api for a quick prototype to see if it catches on as a useful feature before spinning up your own multimodal TTS: we have worked with this openai recently.

https://github.com/openai/openai-realtime-api-beta

the key these though is making sure u control interruptions well at your llm responses and audio stream processing layer
(separate clean vocal segments, etc) ... there is a pace in real time voice that works best for the above api.. talking past each other., etc. and other background noise is a harder problem... it doesnt handle that very well unless u take over and dont turn on VAD for their api to do it for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bounty Use this to assign a bounty to an issue enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants