0

Rendezvous Audio filter and music

We are trying to use Randevous to bring in musicians that have to play during a livestream session.  The only issue is that Rendezvous appear to have some audio filter (probably to reduce echo or noise) but this make the sound be like in a phaser or compressor. Is possible to have the original audio without this filter (for example Zoom have this feature called "Original Audio"). 

Thanks 

126replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Ermanno Bonifazi Yes I understand.  I'm just hoping the good people at Wirecast can use that bit of info to help resolve the issue.

    Like
  • Ermanno Bonifazi Rendezvous is using WebRTC technology and a codec that works differently than Zoom. We are exploring how the codec can be modified but we have to examine the resource consequences as well. WebRTC was designed for voice and the codec's behavior is optimized for that by default.

    Like
  • CraigS I appreciate you guys working on this. In my opinion can be relevant for the market in general.

    Like
  • Thank you everyone for bringing up and addressing this—I'm experiencing exactly the same issue using the 'remote guest' feature in Livestream/Vimeo's Studio 6 platform. I was hoping to switch over to Wirecast if they did not have the same issue, but alas it is exactly the same.. if you were to fix this I would gladly switch to Wirecast, as I find the program more intuitive to use

    Like
  • Update—I did some research and see that Cleanfeed (https://cleanfeed.net/faq)—a very high quality web-based multi-track recorder, uses the Opus codec (which is a mandatory codec for WebRTC). If they can have such high quality audio using WebRTC, surely it's possible?

    Like 1
  • Yes, that would be awsome. I am struggling with the same issue and as a musician most we care about is sound. It would definitely be a game changer and cleanfeed shows it is possible. Should we open another case? 

    Like
      • CraigSModerator
      • Telestream Desktop Forum Moderator
      • CraigS
      • 7 mths ago
      • Reported - view

      Clemens Teufel  need to hear from you as well to message you the instructions for testing.

      Like
  • Christos Vayens This is very interesting Christos. Will give a try.

    Like
  • Ermanno Bonifazi Hi Ermanno fyi Cleanfeed is an audio-only web-based platform, but both you and your guest could technically sign onto Cleanfeed and then using an internal audio router like loopback (https://rogueamoeba.com/loopback/) assign the Cleanfeed audio source from your web browser to the audio source of your rendezvous guest in Wirecast.. but this is of course not at all ideal, and complicates things by running 2 streaming programs at once (we've tried this, audio sounds great but the whole thing is very clunky and there's lots of audio/video lag to address). But my main point is that if Wirecast simply utilized the Opus Codec (which they may already) and tweak it to improve audio quality, this is theoretically possible (see more about Opus here: https://webrtcglossary.com/opus/)

    Like
  • CraigS said:
    Rendezvous is using WebRTC technology and a codec that works differently than Zoom. We are exploring how the codec can be modified but we have to examine the resource consequences as well. WebRTC was designed for voice and the codec's behavior is optimized for that by default.

    Hi CraigS

    I have a feeling that the problem is not codec related. I have used another WebRTC system that has a similar setting to zoom - to use original unprocessed audio. From what I could tell, its basically turning off echo suppression. Its an assumption, but I think that the WebRTC libraries probably have an echo suppression feature that can be toggled during call setup. If that's the case, we just need to have that exposed in the Rendezvous UI... Could one of your engineering people confirm if there is such a parameter in the underlying libraries that could be toggled?

    Greg Kuhnert

    Like
  • Greg This was my proposal as well. I also have researched and learned about the Opus codec too after  Christos Vayens mention, (maybe this is what RV is already using just is its narrowband form) and I think that adding that as selectable option for Stereo Fullband will also be appealing. https://webrtcglossary.com/opus/

    Like
  • This is a little old but very informative. The demo at 28' is quite amazing. https://www.youtube.com/watch?v=iaAD71h9gDU

    Like
  • Ermanno Bonifazi Regarding Opus - It might be interesting to look at. However, lets fix the echo suppression first. Thats a known issue, and it is a client side change only - Just a setting to send to the end browsers in terms of configuration.

    As for codecs, there are lots of more complicated aspects to consider beyond a parameter. Will every browser support it? What are the performance characteristics? How nicely does it play with other elements? Sync issues? How does it work via a proxy like the Telestream turn servers? What is the latency? At face value, a simple change. We all want the outcome of better audio, but lets start with the echo suppression option. That will fix a lot of things.

    Greg Kuhnert

    Like
  • Greg Kuhnert Since they are using WebRTC  for RV and opus is part of WebRTC all the answer are yes, like exactly right now. Latency for Opus is 8ms, lot less than any other Codec.

    Like
  • Thanks for putting time in to do research Ermanno Bonifazi , Greg Kuhnert if you watch the video Ermanno shared it addresses a lot of your questions. Although disabling echo suppression (or whatever compression/noise limiting is going on) would certainly be an improvement, to be honest if Wirecast is using the other WebRTC codec (G.711) even having an 'original sound' option similar to Zoom will probably leave a lot to be desired if you are producing live musical events. My organization has hosted over 50 streaming classical music concerts since social distancing began and we've found that using a platform such as Zoom is just not up to the task, event with 'original sound' enabled. Opus actually appears to be an extremely versatile codec; it has the lowest latency and gives you far superior audio quality even at lower bit rates. Browser compatibility would be identical. CraigS  I know this is probably a big ask for Telestream, but it also seems like an inevitable necessity in the evolution of live streaming production. (also my previous reply seems to still be under pending review, perhaps because I edited it a few times, so I had to create another account to reply here. hope I'm not disrupting things by sharing TMI :)

    Like
  • Christos Vayenas Lets read what the user guide says for Zoom:

    Original sound allows you to preserve the sound from your microphone without using Zoom's echo cancellation and audio-enhancing features. This is ideal if your microphone or sound equipment has these features built-in and you do not need the additional enhancement. 

    Like I said before, lets start with the basics before we look at changing codecs. I've worked in the telecommunications industry all my life, having worked as an architect in building the worlds largest Asterisk VOIP network. So, I am quite familiar with codecs. The reality remains that echo suppression with opus will suck just as bad. Lets walk before we fly. Fix the easy one first... You also talk about IF Wirecast is using 711. Is it? I don't know to be honest. I have not reverse engineered that aspect... but again, lets start with step 1.

    Greg Kuhnert

    Like
  • Ermanno Bonifazi By the way - What codec does Zoom use? You said it worked fine? As per the notes in the zoom documentation, the only change is to turn off echo suppression. Thats my point again and again. Lets fix what we know to be a problem, before we try to find solutions to an unknown problem.

    Greg Kuhnert

    Like
  • Greg Kuhnert I don't think that original sound in Zoom remove only Echo suppression, because they also have "Suppress persistent Background Noise" and "Suppress Intermittent Background Noise" and those are the one exposed on the UI, so maybe there are more embedded in the code. For sure activating the "Original Audio" remove those and I assume any other "non exposed" filters.  

    This is what I found about the codec used by Zoom but there are not detailed info about the way they use or configure it (is not open source) : 

    Zoom Audio codecs supported:

    • G.711
    • G.722
    • OPUS

    Note: OPUS is currently only supported with SIP connections.

    This informations anyway are related to Zoom Room connector, so is not clear if the Zoom client use these codecs too and how. In another interesting document is mentioned their own proprietary WASM codec transferred over WebRTC channel (https://webrtchacks.com/zoom-avoids-using-webrtc/ )

    This let me "presume" that Zoom is using its own "version" of WebRTC implementation as well but ...who knows, there is not any real technical documentation at this level.

    Is Zoom HiFi quality (like Christos Vayenas  was hoping for classic?) No, but for sure is far superior (see my video than RV and even Skype) for now. Hoping for something amazing for RV in the future.

    Like
  • Greg Kuhnert completely agree that no matter what codec is being used we need an option to disable all filters. Zoom is most likely using one of the G.7xx codecs in their normal platform; it works fine with all filters removed in some musical instances but in my experience can't handle the fidelity required for acoustic classical music. Even the way it is described in the quoted piece above implies that the noise cancellation features (whether provided by zoom or by the user) are meant to support speech, not musical performance. This is why I feel like it must be using a G.7xx codec, as these are designed for speech, not music. Looks like the OPUS codec is available for use, but only with VOIP users. If you want to compare the audio quality of Zoom with a platform using OPUS, look into Stage Ten—it's a WebRTC streaming platform that's production features don't hold a candle to Wirecast, but I've been forced to use for events where I have to patch in remote musicians as it is the audio is far superior to any other platform offering this option. The developer actually just confirmed with me an hour ago that they use the OPUS codec. This proves that it is not only possible, but that a much more basic (free or $20/mo) platform is using it right now and offering far superior sound than high end programs like Wirecast and Studio 6 for their remote guest options. Again, if a basic next gen platform like Stage Ten is already using these features, the high end platforms need to catch up or they will be left behind

    Like
  • Christos Vayenas I couldn't agree more and I just can't imagine why they would want to miss out on that part making them less competitive even though it is the much better platform overall. 

    Like
  • Thanks Clemens Teufel , all of this is of course very new and so I'm sure these issues are just now being brought to their attention. but hopefully threads like this attract attention and inspire them to explore implementing these changes

    Like
  • Christos Vayenas and btw as development effort can be broken in stages:

    a) option to remove all filter with current codec (if helps)

    b) option for new codec and related options

    But this is their business, and if they want give us all together (not in years), we will appreciate :)

    Like
  • Greg Kuhnert said:
    How nicely does it play with other elements?

     Rendezvous uses OPUS as should be expected but the above comment is what we must investigate.

    Like 2
  • CraigS  that's very exciting news, looking forward to hearing what the findings are

    Like
  • Christos Vayenas 

    CraigS said:
    Rendezvous uses OPUS

    So. Again, I go back to my earlier point. We need to fix known problems, not unknown problems. Browsers send SDP packets with a createOffer that specifies the voice codec. A quick search confirmed that Chrome will list Opus as its most preferred codec. We know Wirecast supports Opus. So lets put that argument aside. Its not relevant. The original conversation went off the rails, with the belief that "it's a limitation of the WebRTC codec in use". The statement above has put that to bed. Its totally unrelated.

    Lets get back to the original request:

    The only issue is that Rendezvous appear to have some audio filter (probably to reduce echo or noise) but this make the sound be like in a phaser or compressor. Is possible to have the original audio without this filter (for example Zoom have this feature called "Original Audio"). 

    The original poster wants to use the same capability that is available in Zoom. I've checked the zoom documentation, and all they do is turn off echo cancellation and auto gain control from memory.

    We know that by default WebRTC will enable echo cancellation, noise suppression, and autogaincontrol.

        autoGainControl: false,
        echoCancellation: false,
        noiseSuppression: false,

    Those few SDP constraint elements will probably be all that is required. But for the sake of argument, the original poster has specified the fact that Zoom audio with this switch enabled is a desired outcome. It would be trivial to reverse engineer with a packet capture the specific SDP flags used for that solution, and replicate that in rendezvous. Lets start there, instead of focussing on the known problem, instead of speculating about elements that are not part of the problem space.

    Greg Kuhnert

    Like 1
  • Yes Greg Kuhnert I saw that and responded, thank you for your further research. Just to be clear for the work I do neither an 'original sound' Zoom codec, nor an 'original sound' OPUS codec that only utilizes it's narrow band settings will be enough for my needs; for me it's very important that OPUS' full potential as a codec be utilized and optimized for MUSIC in Wirecast. This is why I keep pushing this point. Otherwise it get's me no closer to the sound I'm looking for than using narrow bandwidth original sound in Zoom.

    Like
Like Follow
  • 1 mth agoLast active
  • 126Replies
  • 668Views
  • 19 Following