Skip to content

Conversation

rahulrangers
Copy link
Contributor

Implemented ProcessBuilder-based process spawning for Scala Native using posix_spawn, with support for stdin, stdout, and stderr piping. Integrated pidfd_open and fileDescriptorPoller to asynchronously and non-blockingly wait for process termination. Falls back to waitpid when pidfd_open is unavailable.

@rahulrangers
Copy link
Contributor Author

@armanbilge can you please review this PR

Copy link
Member

@armanbilge armanbilge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice work! A few quick thoughts:

  1. Let's try to get CI green. The native bindings you introduced may need @nowarn annotations (there should be other examples in the codebase).
  2. I think we can unguard some ProcessSuite tests, at least on Linux.
  3. We may want to keep the java.lang.Process implementation in shared sources, and use it as a fallback on Native for non Linux/macOS (i.e. Windows).

@antoniojimeneznieto
Copy link

I checked the timeout exception in the env inheritance test. I think the issue comes in how the envMap/envp is being constructed. Interestingly, adding a dummy variable when the Map is empty seems to solve the problem.

@rahulrangers
Copy link
Contributor Author

@antoniojimeneznieto Yep, I tried using dummy variable, and it actually resolved the issue.
However, using dummy variable doesn't result in countEnv(false) == 0 as it does on the JVM platform. Instead, we get countEnv(true) > countEnv(false), which still causes the test to pass.
So I’m unsure if this is the correct fix, since the behavior isn’t exactly the same.

Also, I’m not fully understanding why the EOF signal isn’t being triggered in readFd, which is preventing the loop from exiting. Currently, both test cases failing in CI is due to this issue.

@antoniojimeneznieto
Copy link

Yes! this isn't a fix, since it may introduce other issues or conflicts. It was more of a remark than a proper solution, sorry for the confusion.

@rahulrangers
Copy link
Contributor Author

rahulrangers commented Jun 9, 2025

@armanbilge @antoniojimeneznieto I have found the issue, The epollSystem was not including EPOLLHUP, which is necessary to detect when the pipe has been closed by a process. This is the reason why the testcases are failing.
I have made a PR to handle EPOLLHUP typelevel/cats-effect#4422

@rahulrangers rahulrangers requested a review from armanbilge June 9, 2025 14:17
@rahulrangers rahulrangers marked this pull request as draft July 9, 2025 13:04
@rahulrangers
Copy link
Contributor Author

@armanbilge can you please review this PR

@rahulrangers rahulrangers marked this pull request as ready for review July 12, 2025 16:29
Copy link
Member

@armanbilge armanbilge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry it took me so long to give this the review it deserves! Overall this looks very good and so happy to seem so many tests enabled in the test suite :) great job with that

I do have a quite a bit of feedback but it is mostly polish. Also, a general advice is to use more of the helpers like guard_ and errnoToThrowable in NativeUtil.


@extern
@nowarn212("cat=unused")
object SyscallBindings {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
object SyscallBindings {
private object syssyscall {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can move the bindings to fs2/io/native/src/main/scala/fs2/io/internal

@inline private def closeAll(fds: Int*): Unit =
fds.foreach { fd => close(fd); () }

def forAsync[F[_]: LiftIO](implicit F: Async[F]): Processes[F] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def forAsync[F[_]: LiftIO](implicit F: Async[F]): Processes[F] =
def forLiftIO[F[_]: LiftIO](implicit F: Async[F]): Processes[F] =

} else {
val status = stackalloc[CInt]()
waitpid(proc.pid, status, WNOHANG)
()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this else? Since it doesn't seem to use the status.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are having else to remove the zombie process from the kernel space as given in this:
A child that terminates, but has not been waited for becomes a
"zombie".
https://man7.org/linux/man-pages/man2/waitpid.2.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants