Merge pull request #413 from sidney-tio/update-custom-nodes

Update custom nodes guide

docs/source/getting_started/03_custom_nodes.md

@@ -1,8 +1,8 @@
-# Guide: Building Custom Nodes to run with PeekingDuck
+# Building Custom Nodes to run with PeekingDuck
 
 PeekingDuck is designed to work with custom use cases. This guide will showcase how anyone can develop custom nodes to be used with PeekingDuck.
 
-In this tutorial, we'll be building a custom csv_logger which writes data into a file. It serves as a data collection step that stores useful metrics into a csv file. This csv file can be used for logging purposes and can serve as a base for data analytics.
+In this tutorial, we'll be building a custom `csv_logger` which writes data into a file. It serves as a data collection step that stores useful metrics into a csv file. This csv file can be used for logging purposes and can serve as a base for data analytics. (PeekingDuck already has an [`output.csv_logger`](/peekingduck.pipeline.nodes.output.csv_writer.Node) node, but this is a good example for users to experience creating a custom node from scratch.)
 
 A step by step instruction to build this custom node is provided below.
 
@@ -22,30 +22,25 @@ Once PeekingDuck has been installed, initialise a new project with our template
 > mkdir results
 ```
 
-### Folder Structure
-Your project folder should look like this at the end of the tutorial.
+Your project folder should look like this:
 
 ```bash
 .
 ├── src
 │   └── <custom_folder_name>
-│       ├── <node_type>               # <node_type> = "output" in this tutorial
-│       │    ├── utils
-│       │    │   └── csv.py
-│       │    └── csv_writer.py
 │       └── configs
-│            └── <node_type>          # <node_type> = "output" in this tutorial
-│                └── csv_writer.yml
 ├── run_config.yml
 ├── results
 ```
 
 - `src/<custom_folder_name>` is where custom nodes created for the project should be housed.
 - `run_config.yml` is the basic yaml file to select nodes in the pipeline. You will be using this to run your peekingduck pipeline.
 
-## Step 2: Populate run_config
+## Step 2: Populate run_config.yml
+
+The `run_config.yml` will be created by `peekingduck init`. For this tutorial, replace its contents with the following node list. Notice that newly minted nodes are identified by the following structure `<custom_folder_name>.<node-type>.<node>`. In this project, the `<custom_folder_name>` is named `custom_nodes`.
 
-The `run_config.yml` will be created. Input the following lines of code into the file. Newly minted nodes needs to be identified by the following structure `<custom_folder_name>.<node-type>.<node>`. In this project, the `<custom_folder_name>` is named `custom_nodes`.
+**IMPORTANT**: `<node_type>` needs to adhere to one of the five node types in PeekingDuck: `['input','output','model','dabble','draw']`
 
 
 ```yaml
@@ -55,24 +50,24 @@ nodes:
 - model.yolo
 - dabble.bbox_count
 - draw.bbox
-- draw.bbox_count
+- draw.legend
 - output.screen
 - custom_nodes.output.csv_writer
 ```
 
 PeekingDuck is designed for flexibility and coherence between in-built and custom nodes; you can design a pipeline which has both types of nodes.
 
-As PeekingDuck runs the pipeline sequentially, it is important to check if the nodes preceding your custom nodes provides the correct inputs to your node.
+As PeekingDuck runs the pipeline sequentially, it is important to check if the nodes preceding your custom nodes provides the correct inputs to your node. PeekingDuck will return an error if the sequence of nodes are incorrect.
 
 ## Step 3: Create Node Configs
 
 Create a custom node config under `src/<custom_folder_name>/configs/<node_type>/<node>.yml`. In this tutorial it will be `src/custom_nodes/configs/output/csv_writer.yml` to list configurations required by the system.
 
 ```yaml
-input: ["count"]                         # inputs required by the node
-output: ["end"]                          # outputs of the node
-period: 1                                # time interval (s) between logs
-filepath: 'results/stats.csv'            # output file location
+input: ["count"]                # inputs required by the node
+output: ["none"]                # outputs of the node.
+period: 1                       # time interval (s) between logs
+filepath: 'results/stats.csv'   # output file location
 ```
 
 Node configs contains information on the input and outputs for PeekingDuck to manage.
@@ -81,7 +76,7 @@ We recommend new users to use the [config template](https://github.com/aimakersp
 Your node config yaml file should contain the following:
 - `input` (list of str): the key(s) to the required inputs for your node
 - `output` (list of str): the key(s) to the outputs
-- (optional) node-specific parameters. In our case, it will be `period`.
+- (optional) node-specific parameters. In our case, it will be `period` and `filepath`.
 
 Note: While the keys for input and output can be arbitrary strings, keys should be consistent across all nodes in the pipeline.
 
@@ -97,7 +92,7 @@ We recommend new users to use the node template below.
 from peekingduck.pipeline.nodes.node import AbstractNode
 
 class Node(AbstractNode):
-    def __init__(self, config):
+    def __init__(self, config = None, **kwargs):
         super().__init__(config, node_path =__name__)
 
     def run(self, inputs):
@@ -113,20 +108,20 @@ from peekingduck.pipeline.nodes.node import AbstractNode
 from .utils.csv import CSVLogger
 
 class Node(AbstractNode):
-    """Node that draws object counting"""
+    """Node that logs outputs of PeekingDuck and writes to a CSV"""
 
-    def __init__(self, config: Dict[str, Any]) -> None:
-        super().__init__(config, node_path=__name__)
-        file_path = config["filepath"]
-        inputs = config["input"].copy()
-        period = config["period"]
+    def __init__(self, config: Dict[str, Any] = None, **kwargs: Any) -> None:
+        super().__init__(config, node_path=__name__, **kwargs)
+        file_path = self.filepath
+        inputs = self.input.copy()
+        period = self.period
         self.csv_logger = CSVLogger(file_path, inputs, period)
 
     def run(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
         """Method that draws the count on the top left corner of image
 
          Args:
-             inputs (dict): Dict with keys "count" and "img".
+             inputs (dict): Dict with keys "count".
          Returns:
              outputs (dict): None
          """
@@ -146,7 +141,7 @@ Note:
 
 We recommend placing the utility files together with your node folder `src/<custom_folder_name>/<node_type>/utils/<your-util>.py`. For this tutorial we will place the following code under `src/custom_nodes/output/utils/csv.py`.
 
-The implementation below uses `period` which dictates the time interval (in seconds) between each log entry.
+The implementation below uses `period` (declared in your configs) to `file_path` (also in configs) which dictates the time interval (in seconds) between each log entry.
 
 ```python
 # csv.py
@@ -185,19 +180,35 @@ class CSVLogger:
 
 ### Final Checks
 
-- By default, PeekingDuck assumes that your custom nodes are found in `src/<custom_folder_name>`.
+- [ ] By default, PeekingDuck assumes that your custom nodes are found in `src/<custom_folder_name>`.
+- [ ] NOTE: Create a `results` folder at the same level as `src`!
+- [ ] Ensure that the files are in the correct folder structure.
 
-- Ensure that the files are in the correct [folder structure](#folder-structure).
 
-- NOTE: Create a `results` folder at the same level as `src`!
+Your repository should look similar to this:
+
+```bash
+.
+├── src
+│   └── <custom_folder_name>
+│       ├── output
+│       │    ├── utils
+│       │    │   └── csv.py
+│       │    └── csv_writer.py
+│       └── configs
+│            └── output
+│                └── csv_writer.yml
+├── run_config.yml
+├── results
+```
 
 ### Running the Module
 
 - Now the setup is complete and ready to run! Run the setup using the following code.
 
     `peekingduck run --config_path run_config.yml`
 
-- If the setup is working, you should see an output screen being displayed. Terminate the program by clicking on the output screen and pressing `q`.
+- If the setup is working, you should see an output screen being displayed. If you are using `input.live`, terminate the program by clicking on the output screen and pressing `q`.
 
 - After the program finishes running, open the file at `results/stats.csv` to view the stored information.
 

peekingduck/cli.py

@@ -48,9 +48,11 @@ def create_custom_folder(custom_folder_name: str) -> None:
     """
     curdir = _get_cwd()
     custom_folder_dir = os.path.join(curdir, "src", custom_folder_name)
+    custom_configs_dir = os.path.join(custom_folder_dir, 'configs')
 
     logger.info("Creating custom nodes folder in %s", custom_folder_dir)
     os.makedirs(custom_folder_dir, exist_ok=True)
+    os.makedirs(custom_configs_dir, exist_ok=True)
 
 
 def create_yml() -> None: