Powershell command not working: removing lines from file1 that are in file2

I am using the following code to remove lines from file1.txt that are in file2.txt.

powershell -Command "$(Get-Content file1.txt) | Where-Object {$_ -notIn $(Get-Content file2.txt)}"

But I'm getting an error regarding -notIn, looking for a value expression. But file2.txt does exist and is not null.

What is causing the error, and how to fix it?

2 answers

  • answered 2017-06-17 19:38 LotPings

    For PowerShell v2 reverse the arguments and use -NotContains

    powershell -Command "$(Get-Content file1.txt) | Where-Object {$(Get-Content file2.txt) -NotContains $_ }
    

    Reference

  • answered 2017-06-17 19:38 mklement0

    To complement LotPings' helpful answer:

    • For execution speed, do not execute Get-Content file2.txt in every loop iteration - cache its result beforehand.

    • For memory efficiency, do not collect all lines of file1.txt up front with $(...), before sending them through the pipeline.

    Therefore (I'm omitting the powershell -Command "..." wrapper for brevity):

    $f2 = Get-Content file2.txt; Get-Content file1.txt | Where-Object { $f2 -NotContains $_ }
    

    Which $ are necessary, and why?

    (...) and $(...) (the subexpression operator) and @(...) (the array subexpression operator) all collect the entire command's / commands' output as a whole, in an array ([System.Object[]]), except if the output comprises only 1 item, in which that item itself is returned.

    (...), which can only enclose a single command, is needed:

    • to clarify precedence in an expression,
    • to use a command (cmdlet / function / external utility call ) as part of an expression.
    • In a pipeline, you can use it to force collecting the enclosed command's entire output beforehand, but doing so negates the memory-throttling benefit of a pipeline.
      • That said, using something like (Get-Content file) | ... | Set-Content file enables updating a given file file "in-place" - but do note that this requires the entire contents of file to fit into memory.
    • Unless you have additional requirements as stated below, prefer (...) to $(...) and @(...).

    $(...) is only needed:

    • to treat multiple commands as a unit,

    • to embed one or more commands inside "..." (a double-quoted string, whose contents are subject to interpolation (expansion)).
      Among the operators listed, $(...) is the only one that can (directly) be embedded inside a double-quoted string.

    @(...) is only needed:

    • to ensure that a command's output is an array, even if only 1 item is returned.

    • In PSv3+, the unified handling of scalars and arrays typically makes @(...) unnecessary, however - see this answer of mine.