Feature/remove write and send to (#108)

* remove `write-to` and `send-to` and rename `print` to `search` on alerts/security-data cmds

* remove unused loggers

* update tests

* fix case where exception is printed on KeyboardInterrupt and closed pipe.

* update readme and changelog

* add TCP examples

* remove tests for removed func

* remove tests for removed loggers

* fix bug from change in latest py42

* add note about external tools for more complex requirements

* updated docs

Author: timabrmsn

CHANGELOG.md

@@ -13,16 +13,23 @@ how a consumer would use the library (e.g. adding unit tests, updating documenta
 ### Changed
 
 - `-i` (`--incremental`) has been removed, use `-c` (`--use-checkpoint`) with a string name for the checkpoint instead.
+
 - The code42cli has been migrated to the [click](https://click.palletsprojects.com) framework. This brings:
-    - BREAKING CHANGE: Commands that accept multiple values for the same option now must have the option flag provided 
-        before each value: 
-            `--option value1 --option value2` instead of `--option value1 value2` (which was previously possible).
+    - BREAKING CHANGE: Commands that accept multiple values for the same option now must have the option flag provided before each value: 
+        use `--option value1 --option value2` instead of `--option value1 value2` (which was previously possible).
     - Cosmetic changes to error messages, progress bars, and help message formatting.
 
+- The `print` command on the `security-data` and `alerts` command groups has been replaced with the `search` command. 
+    This was a name change only, all other functionality remains the same.
+
 ### Added
 
 - Profile can now save multiple alert and file event checkpoints. The name of the checkpoint to be used for a given query should be passed to `-c` (`--use-checkpoint`).
 
+### Removed
+
+- The `write-to` and `send-to` commands on `security-data` and `alerts` command groups. 
+
 ## 0.7.3 - 2020-06-23
 
 ### Fixed

README.md

@@ -52,7 +52,7 @@ When the `--profile` flag is available on other commands, such as those in `secu
 instead of the default one. For example,
 
 ```bash
-code42 security-data print -b 2020-02-02 --profile MY_SECOND_PROFILE
+code42 security-data search -b 2020-02-02 --profile MY_SECOND_PROFILE
 ```
 
 To see all your profiles, do:
@@ -63,19 +63,18 @@ code42 profile list
 
 ## Security Data and Alerts
 
-Using the CLI, you can query for security events and alerts and send them to three possible destination types:
+Using the CLI, you can query for security events and alerts just like in the admin console, but the results are output
+to stdout so they can be written to a file or piped out to another process (for sending to an external syslog server, for 
+example). 
 
-* stdout
-* A file
-* A server, such as SysLog
 
 The following examples pertain to security events, but can also be used for alerts by replacing `security-data` with 
 `alerts`:
 
 To print events to stdout, do:
 
 ```bash
-code42 security-data print -b <begin_date>
+code42 security-data search -b <begin_date>
 ```
 
 Note that `-b` or `--begin` is usually required.
@@ -85,79 +84,110 @@ And end date can also be given with `-e` or `--end` to query for a specific date
 To specify a begin/end time, you can pass a date or a date w/ time as a string:
 
 ```bash
-code42 security-data print -b '2020-02-02 12:51:00'
+code42 security-data search -b '2020-02-02 12:51:00'
 ```
 
 ```bash
-code42 security-data print -b '2020-02-02 12:30'
+code42 security-data search -b '2020-02-02 12:30'
 ```
 
 ```bash
-code42 security-data print -b '2020-02-02 12'
+code42 security-data search -b '2020-02-02 12'
 ```
 
 ```bash
-code42 security-data print -b 2020-02-02
+code42 security-data search -b 2020-02-02
 ```
 
 or a shorthand string specifying either days, hours, or minutes back from the current time:
 
 ```bash
-code42 security-data print -b 30d
+code42 security-data search -b 30d
 ```
 
 ```bash
-code42 security-data print -b 10d -e 12h
+code42 security-data search -b 10d -e 12h
 ```
 
-Begin date will be ignored if provided on subsequent queries using `-i`.
+Begin date will be ignored if provided on subsequent queries using `-c/--use-checkpoint`.
 
 Use different format with `-f`:
 
 ```bash
-code42 security-data print -b 2020-02-02 -f CEF
+code42 security-data search -b 2020-02-02 -f CEF
 ```
 
 The available formats are CEF, JSON, and RAW-JSON.
 Currently, CEF format is only supported for security events.
 
-To write events to a file, do:
+To write events to a file, just redirect your output:
 
 ```bash
-code42 security-data write-to filename.txt -b 2020-02-02
+code42 security-data search -b 2020-02-02 > filename.txt
 ```
 
-To send events to a server, do:
+To send events to an external server using `netcat` on Linux/Mac:
 
+UDP:
 ```bash
-code42 security-data send-to syslog.company.com -p TCP -b 2020-02-02
+code42 security-data search -b 10d | nc -u syslog.company.com 514 
 ```
 
-To only get events that Code42 previously did not observe since you last recorded a checkpoint, use the `-i` flag.
-
+TCP:
 ```bash
-code42 security-data send-to syslog.company.com -i
+code42 security-data search -b 10d | nc server.company.com 8080
+```
+
+Using `powershell` on Windows:
+
+UDP:
+```powershell
+# set up connection
+$Connection = New-Object System.Net.Sockets.UDPClient("syslog.company.com",514)
+
+# pipe code42 output through connection
+code42 security-data search -b 10d | foreach {$Message = [Text.Encoding]::UTF8.GetBytes($_); $Connection.Send($Message, $Message.Length)}
+```
+
+TCP:
+```powershell
+# set up connection
+$Connection = New-Object System.Net.Sockets.TcpClient("127.0.0.1","65432")
+$Writer = New-Object System.IO.StreamWriter($Connection.GetStream())
+
+# pipe code42 output through connection
+code42 security-data search -b 10d | foreach { $Writer.WriteLine($_); $Writer.Flush() }
 ```
 
-This is only guaranteed if you did not change your query.
+Note: For more complex requirements when sending to an external server (SSL, special formatting, etc.), use a dedicated
+syslog forwarding tool like `rsyslog` or connection tunneling tool like `stunnel`.
+
+If you want to periodically run the same query, but only retrieve the new events each time, use the 
+`-c/--use-checkpoint` option with a name for your checkpoint. This stores the timestamp of the query's last event to a 
+file on disk and uses that as the "begin date" timestamp filter on the next query that uses the same checkpoint name. 
+Checkpoints are stored per profile. 
 
-To send events to a server using a specific profile, do:
+Initial run requires a begin date:
+```bash
+code42 security-data search -b 30d --use-checkpoint my_checkpoint
+```
 
+Subsequent runs do not:
 ```bash
-code42 security-data --profile PROFILE_FOR_RECURRING_JOB send-to syslog.company.com -b 2020-02-02 -f CEF -i
+code42 security-data search --use-checkpoint my_checkpoint
 ```
 
 You can also use wildcard for queries, but note, if they are not in quotes, you may get unexpected behavior.
 
 ```bash
-code42 security-data print --actor "*"
+code42 security-data search --actor "*"
 ```
 
-Each destination-type subcommand shares query parameters
+The search query parameters are as follows:
 
-- `-t` (exposure types)
-- `-b` (begin date)
-- `-e` (end date)
+- `-t/--type` (exposure types)
+- `-b/--begin` (begin date)
+- `-e/--end` (end date)
 - `--c42-username`
 - `--actor`
 - `--md5`
@@ -171,9 +201,29 @@ Each destination-type subcommand shares query parameters
 - `--advanced-query` (raw JSON query)
 
 You cannot use other query parameters if you use `--advanced-query`.
-To learn more about acceptable arguments, add the `-h` flag to `code42` or any of the destination-type subcommands.
+To learn more about acceptable arguments, add the `-h` flag to `code42 security-data` 
+
+Saved Searches:
 
+The CLI can also access "saved searches" that are stored in the admin console, and run them via their saved search ID.
 
+Use the `saved-search list` subcommand to list existing searches with their IDs:
+
+```bash
+code42 security-data saved-search list
+```
+
+The `show` subcommand will give details about the search with the provided ID:
+
+```bash
+code42 security-data saved-search show <ID>
+```
+
+To get the results of a saved search, use the `--saved-search` option with your search ID on the `search` subcommand:
+
+```bash
+code42 security-data search --saved-search <ID>
+```
 
 ## Detection Lists
 
@@ -212,7 +262,9 @@ reported.
 If you keep getting prompted for your password, try resetting with `code42 profile reset-pw`.
 If that doesn't work, delete your credentials file located at ~/.code42cli or the entry in keychain.
 
-## Tab completion
+## Shell tab completion
+
+To enable shell autocomplete when you hit `tab` after the first few characters of a command name, do the following:
 
 For Bash, add this to ~/.bashrc:
 

docs/commands/alerts.md

@@ -1,9 +1,10 @@
 # Alerts
 
-## Shared arguments
+## search
 
-Search args are shared between `print`, `write-to`, and `send-to` commands.
+Search for alerts and print them to stdout.
 
+Arguments:
 * `advanced-query`: A raw JSON alerts query. Useful for when the provided query parameters do not satisfy your 
     requirements. WARNING: Using advanced queries is incompatible with other query-building args.
 * `-b`, `--begin`: The beginning of the date range in which to look for alerts, can be a date/time in yyyy-MM-dd (UTC) 
@@ -32,43 +33,9 @@ Search args are shared between `print`, `write-to`, and `send-to` commands.
 * `-f`, `--format` (optional): The format used for outputting file events. Available choices= [CEF,JSON,RAW-JSON]. 
 * `-c`, `--use-checkpoint` (optional): Get only file events that were not previously retrieved by writing the timestamp of the last event retrieved to a named checkpoint.
 
-## print
-
-Print file events to stdout.
-
-Arguments:
-* search args (note that begin date is often required).
-
-Usage:
-```bash
-code42 alerts print -b <begin-date> <args>
-```
-
-## write-to
-
-Write file events to the file with the given name.
-
-Arguments:
-* `output_file`: The name of the local file to send output to.
-* search args (note that begin date is often required).
-
-Usage:
-```bash
-code42 alerts write-to -b 2020-03-01 <rgs>
-```
-
-## send-to
-
-Send file events to the given server address.
-
-Arguments:
-* `server`: The server address to send output to.
-* `protocol` (optional): Protocol used to send logs to server. Available choices= [TCP, UDP].
-* search args (note that begin date is often required).
-
 Usage:
 ```bash
-code42 alerts send-to <server> <optional-args> <args>
+code42 alerts search -b <begin-date> <options>
 ```
 
 ## clear-checkpoint

docs/commands/securitydata.md

@@ -1,11 +1,14 @@
 # Security Data
 
-## Shared arguments
 
-Search args are shared between `print`, `write-to`, and `send-to` commands.
+## search
 
-* `--advanced-query` (optional): A raw JSON file events query. Useful for when the provided query parameters do not 
+Search for file events and print them to stdout.
+
+Arguments:
+* `--advanced-query` (optional | cannot be used with other query options): A raw JSON file events query. Useful for when the provided query parameters do not 
     satisfy your requirements. WARNING: Using advanced queries is incompatible with other query-building args.
+* `--saved-search` (optional | cannot be used with other query options): Get events from a saved search filter (created in the Code42 admin console) with the given ID.
 * `-b`, `--begin` (required except for non-first runs in checkpoint mode): The beginning of the date range in which to 
     look for file events, can be a date/time in yyyy-MM-dd (UTC) or yyyy-MM-dd HH:MM:SS (UTC+24-hr time) format where 
     the 'time' portion of the string can be partial (e.g. '2020-01-01 12' or '2020-01-01 01:15') or a short value 
@@ -28,45 +31,11 @@ Search args are shared between `print`, `write-to`, and `send-to` commands.
 * `-f`, `--format` (optional): The format used for outputting file events. Available choices= [CEF,JSON,RAW-JSON]. 
 * `-c`, `--use-checkpoint` (optional): Get only file events that were not previously retrieved by writing the timestamp of the last event retrieved to a named checkpoint.
 
-
-## print
-
-Print file events to stdout.
-
-Arguments:
-* search args (note that begin date is often required).
-
-Usage:
-```bash
-code42 security-data print -b <begin-date> <args>
-```
-
-## write-to
-
-Write file events to the file with the given name. 
-
-Arguments:
-* `output_file`: The name of the local file to send output to.
-* search args (note that begin date is often required).
-
 Usage:
 ```bash
-code42 security-data write-to -b 2020-03-01 <args>
+code42 security-data search -b <begin-date> <options>
 ```
 
-## send-to
-
-Send file events to the given server address. 
-
-Arguments:
-* `server`: The server address to send output to.
-* `protocol` (optional): Protocol used to send logs to server. Available choices= [TCP, UDP].
-* search args (note that begin date is often required).
-
-Usage:
-```bash
-code42 security-data send-to <server> <optional-server-args> <args>
-```
 
 ## clear-checkpoint
 

docs/index.md

@@ -13,7 +13,7 @@ To use the Code42 CLI, you must have:
 
 A Code42 Diamond or Platinum product plan 
 Endpoint monitoring enabled in the Code42 console 
-Python version 2.7.x, or 3.5 and later installed
+Python version 3.5 and later installed
 
 ## Content
 

docs/userguides/profile.md

@@ -25,7 +25,7 @@ When the `--profile` flag is available on other commands, such as those in `secu
 instead of the default one. For example,
 
 ```bash
-code42 security-data print -b 2020-02-02 --profile MY_SECOND_PROFILE
+code42 security-data search -b 2020-02-02 --profile MY_SECOND_PROFILE
 ```
 
 To see all your profiles, do: