#266 - experimentation with OS X paths

[instantiator]

Named pipe path length limits on OS X mean that default named pipes that are based in Path.GetTempDir() are too long. This happens when a named pipe is created with just the pipe name. If a full, rooted, path is provided then that is accepted.

The error looks like this:

The path '/var/folders/1c/wq9ptmb53k50bx9dpmh7j8jw0000gn/T/CoreFxPipe_FFMpegCore_411169d3-75c5-4bd8-a8e7-a2045a25540f' is of an invalid length for use with domain sockets on this platform.  The length must be between 1 and 104 characters, inclusive. (Parameter 'path')
Actual value was /var/folders/1c/wq9ptmb53k50bx9dpmh7j8jw0000gn/T/CoreFxPipe_FFMpegCore_411169d3-75c5-4bd8-a8e7-a2045a25540f.

This PR:

• sets a shorter path for named pipes on OS X in PipeHelpers
• uses PipePath in PipeArgument to construct the NamedPipeServerStream

However, although this resolves the issue with the named pipe path length, it leads to another issue.

Test with:

dotnet test --filter "FullyQualifiedName=FFMpegCore.Test.AudioTest.Audio_FromRaw"

This leads to the following exception:

FFMpegCore.Exceptions.FFMpegException: Exception thrown during processing ---> System.Net.Sockets.SocketException: Operation canceled

The SocketException indicates that perhaps the named pipe socket is being closed while it is still in AwaitConnection. Perhaps a connection isn't being properly established. I can't be sure why - although perhaps the issue still lies somewhere in the issue of the pipe name.

[instantiator]

Full stack trace for the operation canceled SocketException:

$ dotnet test --filter "FullyQualifiedName=FFMpegCore.Test.AudioTest.Audio_FromRaw"
  Determining projects to restore...
  All projects are up-to-date for restore.
  FFMpegCore -> /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/bin/Debug/netstandard2.0/FFMpegCore.dll
  FFMpegCore.Test -> /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore.Test/bin/Debug/net5.0/FFMpegCore.Test.dll
Test run for /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore.Test/bin/Debug/net5.0/FFMpegCore.Test.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 16.11.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed Audio_FromRaw [238 ms]
  Error Message:
   Test method FFMpegCore.Test.AudioTest.Audio_FromRaw threw exception:
FFMpegCore.Exceptions.FFMpegException: Exception thrown during processing ---> System.Net.Sockets.SocketException: Operation canceled
  Stack Trace:
      at System.Net.Sockets.Socket.GetException(SocketError error, Boolean wrapExceptionsInIOExceptions)
   at System.Net.Sockets.Socket.CompleteAccept(Socket s, TaskSocketAsyncEventArgs`1 saea)
   at System.Net.Sockets.Socket.<>c.<AcceptAsync>b__281_0(Object s, SocketAsyncEventArgs e)
   at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted(SocketAsyncEventArgs e)
   at System.Net.Sockets.SocketAsyncEventArgs.OnCompletedInternal()
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAsyncFailure(SocketError socketError, Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.CompletionCallback(Int32 bytesTransferred, SocketFlags flags, SocketError socketError)
   at System.Net.Sockets.SocketAsyncEventArgs.AcceptCompletionCallback(IntPtr acceptedFileDescriptor, Byte[] socketAddress, Int32 socketAddressSize, SocketError socketError)
   at System.Net.Sockets.SocketAsyncContext.AcceptOperation.InvokeCallback(Boolean allowPooling)
   at System.Net.Sockets.SocketAsyncContext.AsyncOperation.<>c.<TryCancel>b__18_0(Object o)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
--- End of stack trace from previous location ---
   at System.IO.Pipes.NamedPipeServerStream.<WaitForConnectionAsync>g__WaitForConnectionAsyncCore|24_0()
   at FFMpegCore.Arguments.InputPipeArgument.ProcessDataAsync(CancellationToken token) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/Arguments/InputPipeArgument.cs:line 24
   at FFMpegCore.Arguments.PipeArgument.During(CancellationToken cancellationToken) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/Arguments/PipeArgument.cs:line 45
   at FFMpegCore.FFMpegArguments.During(CancellationToken cancellationToken) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArguments.cs:line 73
   at FFMpegCore.FFMpegArgumentProcessor.Process(Instance instance, CancellationTokenSource cancellationTokenSource) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArgumentProcessor.cs:line 154
   at FFMpegCore.FFMpegArgumentProcessor.ProcessAsynchronously(Boolean throwOnError, FFOptions ffMpegOptions) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArgumentProcessor.cs:line 135
--- End of inner exception stack trace ---
    at FFMpegCore.FFMpegArgumentProcessor.HandleException(Boolean throwOnError, Exception e, IReadOnlyList`1 errorData) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArgumentProcessor.cs:line 215
   at FFMpegCore.FFMpegArgumentProcessor.ProcessAsynchronously(Boolean throwOnError, FFOptions ffMpegOptions) in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArgumentProcessor.cs:line 139
   at FFMpegCore.Test.AudioTest.Audio_FromRaw() in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore.Test/AudioTest.cs:line 46
   at FFMpegCore.Test.AudioTest.Audio_FromRaw() in /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore.Test/AudioTest.cs:line 46
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ThreadOperations.ExecuteWithAbortSafety(Action action)

  Standard Output Messages:

 Debug Trace:
 Disposing NamedPipeServerStream on InputPipeArgument
 Disposing NamedPipeServerStream on OutputPipeArgument
 Disconnecting NamedPipeServerStream on InputPipeArgument
 Disconnecting NamedPipeServerStream on OutputPipeArgument

Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 262 ms - /Users/lewiswestbury/src/personal/FFMpegCore/FFMpegCore.Test/bin/Debug/net5.0/FFMpegCore.Test.dll (net5.0)

[rosenbjerg]

@instantiator Which ,NET version did you experiment with? Just curious if any of this got fixed with the .NET 6 release for Mac OS

[ArthurD]

Still have this issue (path exceeds the 104 char limit) on .NET 6 when on macOS (running the latest version of macOS).

[oleksii-mykhniak]

I also have the same problem on .Net 6 and on macOS v12.3.1 (the latest one at this moment).
FFMpegCore v4.8.0 also I tried xFFmpeg.NET v7.1.3 and had the same error, could be a problem with ffmpeg version 5.0.1?

    [HttpPost("Convert")]
    [DisableRequestSizeLimit]
    public async Task ConvertAndUpload(IFormFile file)
    {
        var inputStream = new MemoryStream();
        await file.CopyToAsync(inputStream);
        inputStream.Position = 0;

        GlobalFFOptions.Configure(new FFOptions { BinaryFolder = "/Users/user/Downloads/", TemporaryFilesFolder = "/tmp" });

        var memoryStream = new MemoryStream();
        await FFMpegArguments
            .FromPipeInput(new StreamPipeSource(inputStream))
            .OutputToPipe(new StreamPipeSink(memoryStream), options => options
                .WithAudioSamplingRate(44100)
                .WithVideoFilters(vo => vo.Scale(FFMpegCore.Enums.VideoSize.FullHd))
                .ForceFormat("m4v")
            )
            .ProcessAsynchronously();

         // more code
      }

[krauthaufen]

Same problem here, would really like to see that fixed...

[yeroc-sebrof]

Hi Team, same problem here on MacOS M1. I'm working on a ASP.NET Core 3.1 service and was attempting to run FFProbe against a .MOV file:

Exception thrown: 'System.ArgumentOutOfRangeException' in System.Private.CoreLib.dll
Exception thrown: 'System.ArgumentOutOfRangeException' in System.Private.CoreLib.dll
MDS.Controllers.MediaController: Error: Error: Could not save media - The path '/var/folders/z7/ysq5ds5n5ljfxj3dw671d9240000gn/T/CoreFxPipe_FFMpegCore_d23bfbe7-314e-457e-bfa0-e81cf8732a98' is of an invalid length for use with domain sockets on this platform.  The length must be between 1 and 104 characters, inclusive. (Parameter 'path')
Actual value was /var/folders/z7/ysq5ds5n5ljfxj3dw671d9240000gn/T/CoreFxPipe_FFMpegCore_d23bfbe7-314e-457e-bfa0-e81cf8732a98.
�[41m�[30mfail�[39m�[22m�[49m: MDS.Controllers.MediaController[0]
      Error: Could not save media - The path '/var/folders/z7/ysq5ds5n5ljfxj3dw671d9240000gn/T/CoreFxPipe_FFMpegCore_d23bfbe7-314e-457e-bfa0-e81cf8732a98' is of an invalid length for use with domain sockets on this platform.  The length must be between 1 and 104 characters, inclusive. (Parameter 'path')
      Actual value was /var/folders/z7/ysq5ds5n5ljfxj3dw671d9240000gn/T/CoreFxPipe_FFMpegCore_d23bfbe7-314e-457e-bfa0-e81cf8732a98.

[dylanbeattie]

I'm having the same issue here (.NET 6.0.202, macOS Monterey 12.3.1, M1 Macbook Pro) - but only when I run as a regular user:

$ dotnet run
Unhandled exception. System.ArgumentOutOfRangeException: The path 
'/var/folders/7w/_s8y6ppx4dj_q3glhw02y31w0000gn/T/CoreFxPipe_FFMpegCore_504df149-9ed2-4ca7-9153-ce3657856376' 
is of an invalid length for use with domain sockets on this platform.  
The length must be between 1 and 104 characters, inclusive. (Parameter 'path')

If I sudo run the exact same code:

$ sudo dotnet run
Done

The actual path being returned from GetPipePath is unix:/tmp/CoreFxPipe_FFMpegCore_7868133a-9cfb-4eaa-8eac-7c4df2324e93

so it looks like macOS is redirecting the pipe path for non-root users and ending up with something that's too long.

If I modify the GetUniquePipename method to return a shorter name:

public static string GetUnqiuePipeName() => $"FFMpegCore_{Guid.NewGuid().ToString("N")}";

then I get this instead:

unix:/tmp/CoreFxPipe_FFMpegCore_5b78210b9ca64e908244528d23bf4710: No such file or directory)
   at FFMpegCore.FFMpegArgumentProcessor.HandleCompletion(Boolean throwOnError, Int32 exitCode, IReadOnlyList`1 errorData) in /Users/dylan/Projects/github/dylanbeattie/FFMpegCore/FFMpegCore/FFMpeg/FFMpegArgumentProcessor.cs:line 171

Again, running as sudo works fine.

[krauthaufen]

I think the difference here is that MacOS doesn't accept paths here in user-mode iirc. We had similar problems with shared memory in our project and giving it names instead of paths worked fine. I can dig up the code if anyone wants to know...

[dylanbeattie]

I think the difference here is that MacOS doesn't accept paths here in user-mode iirc. We had similar problems with shared memory in our project and giving it names instead of paths worked fine. I can dig up the code if anyone wants to know...

That would be great; at this point, anything that helps shed light on what's happening and how to fix/work around it would be really appreciated!

[krauthaufen]

Hey,
In our scenario we use a shared memory to get raw image-data from dotnet to an electron browser (which works on MacOS with M1 chips). The dotnet part was sketched here and the real implementation (should be an exact copy but I'm not entirely sure since this was quite some time ago) is here.

Note that we didn't use any of the dotnet standard things (MemoryMappedFile, etc) since they threw exceptions in netcoreapp3.1 (maybe these work with a newer dotnet).

Finally I need to stress that this was about shared memory and not about pipes, but I assumr that these are quite similar in terms of restrictions.

Hope that helps a little

[ep1kt3t0s]

Has this PR still being worked on? There is an issue related to it: #365