Zikai Lin

Algorithms with Leetcode: Backtracking

2022-10-15T00:00:00+00:00

Backtracking

78 Subsets

For each subset, we have every single element whether to include or not. And therefore we have $2^n$ possibilities. And therefore, if we use brutal force, the time complexity is too much.

[1,2,3]

credit to Neetcode.io

Breakdown

At each value of nums, we can divided into two possible cases:

We want to add nums[i] into the result? Or
We DON’T want to add nums[i] into the result.

Therefore, as illustrated in the figure above, starting from the beginning of the array, at each nums[i], we can expand it into two different subtrees (add vs not add). And at the end of the tree, i.e. the leaves, contains all possible subsets that we want to return.

To solve such problems, we can use Depth First Search (DFS), which is commonly used in binary tree structure.

Conclusion

Remember that for backtracking problem, we always want to identify the sub-problems.

What is the sub-problems: include vs NOT include
Is a tree good enough for this type of problems? Yes
DFS vs BFS? DFS

90. Subsets II

Problem Description

Given an integer array nums that may contain duplicates, return all possible subsets (the power set).

The solution set must not contain duplicate subsets. Return the solution in any order.

Example 1:

Input: nums = [1,2,2]
Output: [[],[1],[1,2],[1,2,2],[2],[2,2]]

Example 2:

Input: nums = [0]
Output: [[],[0]]

Constraints:

1 <= nums.length <= 10
-10 <= nums[i] <= 10

39 Combination Sum

Problem description

https://leetcode.com/problems/combination-sum/

Given an array of distinct integers candidates and a target integer target, return a list of all unique combinations of candidates where the chosen numbers sum to target. You may return the combinations in any order.

The same number may be chosen from candidates an unlimited number of times. Two combinations are unique if the frequency of at least one of the chosen numbers is different.

The test cases are generated such that the number of unique combinations that sum up to target is less than 150 combinations for the given input.

Example 1:

Input: candidates = [2,3,6,7], target = 7
Output: [[2,2,3],[7]]
Explanation:
2 and 3 are candidates, and 2 + 2 + 3 = 7. Note that 2 can be used multiple times.
7 is a candidate, and 7 = 7.
These are the only two combinations.

Example 2:

Input: candidates = [2,3,5], target = 8
Output: [[2,2,2,2],[2,3,3],[3,5]]

Example 3:

Input: candidates = [2], target = 1
Output: []

Breakdown

Does brutal force works here? Probably no, because each number can be picked out for unlimited number of times.

So here what is the decision tree?

For each node (each number in the array), expand the whole possible choices as the children.

(Stoppign rule) At each level of the decision tree, how do we determine that if we can move forward or keep searching?

If the path sum is greater or equal than the target, we don’t have to keep searching because all the values in the array are positive

Python solution

class Solution:
    def combinationSum(self, candidates: List[int], target: int) -> List[List[int]]:
        
        res = []
        
        subsum = 0
        subset = []
        
        def dfs(i, target, subsum):
            nonlocal res
            if subsum == target:
                res.append(subset.copy())
                return None
            
            if i >= len(candidates) or subsum > target:
                return None
            
            subset.append(candidates[i])
            dfs(i, target, subsum + candidates[i])
            
            subset.pop()
            dfs(i+1, target, subsum)
            
        dfs(0, target, 0)
        
        return res

40. Combination Sum II

Given a collection of candidate numbers (candidates) and a target number (target), find all unique combinations in candidates where the candidate numbers sum to target.

Each number in candidates may only be used once in the combination.

Note: The solution set must not contain duplicate combinations.

Example 1:

Input: candidates = [10,1,2,7,6,1,5], target = 8
Output: 
[
[1,1,6],
[1,2,5],
[1,7],
[2,6]
]

Example 2:

Input: candidates = [2,5,2,1,2], target = 5
Output: 
[
[1,2,2],
[5]
]

Constraints:

1 <= candidates.length <= 100
1 <= candidates[i] <= 50
1 <= target <= 30

Breakdown

Still, we need to think about what the decision tree is.

Using the example as an illustration:

candidates = [10,1,2,7,6,1,5]

After being sorted: candidates = [1,1,2,5,6,7,10]

Include 1 vs not include 1:
- Include 1 -> create a subproblem where arr = [1,1,2,5,6,7,10], target = 8-1 = 7
- Not include 1 -> Then we have to make sure that the following subtree does not include 1!

Code

class Solution:
    def combinationSum2(self, candidates: List[int], target: int) -> List[List[int]]:
        candidates.sort()
        
        res = []
        cur = []
        def backtrack(cur, pos, target):
            if target == 0:
                # basecase, find a path
                res.append(cur.copy())
            if target <= 0:
                # Stop tracking
                return
            
            
            prev = -1
            for i in range(pos, len(candidates)):
                # Case 2: skip candidates[i]
                if candidates[i] == prev:
                    continue
                
                # Case 2: include candidates[i]
                cur.append(candidates[i])
                backtrack(cur, i+1, target - candidates[i])
                cur.pop() # cleanup
                
                # record prev
                prev = candidates[i]
        backtrack([], 0, target)
        return res
                
                
                
                

77. Combinations

46. Permutations

Given an array nums of distinct integers, return all the possible permutations. You can return the answer in any order.

Example 1:

Input: nums = [1,2,3]
Output: [[1,2,3],[1,3,2],[2,1,3],[2,3,1],[3,1,2],[3,2,1]]

Example 2:

Input: nums = [0,1]
Output: [[0,1],[1,0]]

Example 3:

Input: nums = [1]
Output: [[1]]

Constraints:

1 <= nums.length <= 6
-10 <= nums[i] <= 10
All the integers of nums are unique.

Breakdown

How many possible results exists for permutation?

\[\mathrm{Permute}(n) = n!\]

What does the decision tree look like? (suppose we are using the example)

We pop one number out of [1,2,3]:
- [1,2,3] -> [2,3] + [1]
- [2,3] -> [3] + [2]
- [2,3] -> [2] + [3]
- [1,2,3] -> [1,3] + [2]
  - [1,3] -> [1] + [3]
  - [1,3] -> [3] + [1]
- [1,2,3] -> [1,2] + [3]
  - [1,2] -> [1] + [2]
  - [1，2] -> [2] + [1]
The backtracking problem can therefore be described as:
- Each time, we pop one number out (from 1 to n)
- Run a DFS and sub-divide the array into all possible orders
- After the DFS, we append the number we popped out to it

Solution:

Credit to Neetcode.io

class Solution:
    def permute(self, nums: List[int]) -> List[List[int]]:
        result = []
        
        if (len(nums) == 1):
            return [nums.copy()]
        
        for i in range(len(nums)):
            n = nums.pop(0)
            perms = self.permute(nums)
            
            for perm in perms:
                perm.append(n)
            result.extend(perms)
            nums.append(n)
                
        
        return result
            

Modified Squared Exponential Kernel and Basis Expansion

2022-09-30T00:00:00+00:00

Squared Exponential Kernel

Squared Exponential Kernel, A.K.A, the Radial Basis Function kernel, the Gaussian kernel. It has the form: $k(x,x') = \sigma^2 \dfrac{\|x-x'\|}{2\rho^2}$ It also has only two parameters:

The lengthscale, i.e. (\rho) controls the smoothness of your kernel.
The output variance (\sigma^2) determines the ‘magnitude’ of your covariance function. It is like a scale factor.

Modified Squared Exponential Kernel

We consider modified form of squared-exponential covariance kernel, defined as:

\[\kappa(x, x') = \exp\{-a(\|x\| + \|x'\|^2) - b \|x - x'\|^2\},\quad \text{for }a>0, \quad b> 0,\]

where $\|\cdot\|$ is the $L_2$-norm. Notice that $\kappa(x, x')$ reduces to standard SE kernel when $a = 0$. Here $a$ is the decay parameter controls the decay rate of $\mathrm{Var}$

$\mathrm{Var}\{f_j(\cdot)\}$ compared to $\mathrm{Var}\{f_j(0)\}$. The parameter $b$ is the smoothing parameter of modified SE kernel, smaller value of (b) corresponds to a smoother GP $f_j(\cdot)$. An important property of modified SE kernel is that it has a closed form eigen decomposition using Hermite polynomials. A $k$-th order normalized Hermite polynomials is defined as follow:

\[H_k(x) = (2^k k!\sqrt{x})^{1/2}(-1)^k\exp (x^2)\dfrac{d^k}{dx^k}\exp(-x^2),\]

Mercer’s Theorem states that a positive-definite kernel $k(\mathbf{x},\mathbf{x}')$ in a finite measure space can be decomposed as

\[k(\mathbf{x},\mathbf{x}') = \sum_{i=1}^\infty\lambda_i\phi_{i}(\mathbf{x})\phi_{i}'(\mathbf{x}'),\] \[k(x,x^{'})=\sigma^2\sum^{\infty}_{l=1}\lambda_l\Psi_l(x)\Psi_l(x^{'}) ,\]

where $\{\lambda_l\}^{\infty}_{l=1}$ are eigen values with $\lambda_l\geq\lambda_{l+1}>0$ for $l\geq1$ and $\{\phi_l(x)\}^{\infty}_{l=1}$ are orthonormal eigen functions that satisfy the following properties: $\int\Psi_l(x)\Psi_l(x^{'})dx=1$ when $l=l^{'}$ and $\int\Psi_l(x)\Psi_l(x^{'})dx=0$ otherwise. The GP $f_j(\cdot)$ can then be represented as

\[f(x)=\sum^{\infty}_{l=1}\theta_lb_l(x), \quad \theta_l \overset{i.i.d}{\sim} \mathcal{N}(0,\sigma_l^2), \quad \text{for }l = 1,\ldots, \infty\]

where $b_l(x)=\sqrt{\lambda_l}\phi_l(x)$ is the basis function. In practice, we approximate $f_j(\cdot)$ by a truncated eigen-decomposition using $L$ basis functions, where $L$ is determined by the maximum polynomial degrees, says $h$, i.e. $L = \binom{h+d}{d}$ for $d$-dimensional data.

Fast Basis Function Expansion based on Eigen Decomposition

In R, there’s a nice package called BayesGPfit (Kang) that you can use to do the basis expansion for modified squared-exponential kernel.

Here’s an example, consider using basis functions to approximate a modified squared exponential kernel with $a =0.01$ and $b = 1$:

Reference

Jian Kang (2022). BayesGPfit: Fast Bayesian Gaussian Process Regression Fitting. R package version 1.1.0. https://CRAN.R-project.org/package=BayesGPfit

R-Blog: Heatmap using R Plotly

2021-02-01T00:00:00+00:00

Basics

In plotly, the heatmap can be plotted by the plot_ly function. You can specify the type of plot as heatmap.

library(plotly)
fig <- plot_ly(z = volcano, type = "heatmap")

fig

Now plotly creates a beautiful interactive heatmap visualization for you. However, this is just a simple example without further customization. Sometimes, you might need to specify optional arguments to make your heatmap visualization cleaner and more publishable.

Colors

First let’s start with the color scale. The default color scale in plotly is the Viridis style. But you can either specify a color scheme using the name listed in https://colorbrewer2.org, or a customized colorscale in R. Here are some available color schemes that you can directly use in plotly:

Continuous color scale

Multi-hue colorscale:

Blue-Green colorscale (BuGn)
Blue-Purple colorscale (BuPu)
Green-Blue colorscale (GnBu)
Orange-Red colorscale (OrRd)
Purple-Blue colorscale (PuBu)
Purple-Blue-Green colorscale (PuBuGn)
Purple-Red colorscale (PuRd)
Red-Purple colorscale (RdPu)
Yellow-Green colorscale (YlGn)
Yellow-Green-Blue colorscale (YlGnBu)
Yellow-Orange-Bright colorscale (YlOrBr)
Yellow-Orange-Red colorscale (YlOrRd)

Single hue colorscale:

Blues (Blues)
Greens (Greens)
Greys (Greys)
Oranges (Oranges)
Purples (Purples)
Reds (Reds)

Example - Blues

library(plotly)
fig <- plot_ly(z = volcano, type = "heatmap", colors = "Blues")
fig

Diverging colorscale

Diverging colorscales are often useful when you want to visualize the “sign” in your dataset, e.g. negative-positive. Some existing diverging colorscales available in plotly are as follow:

Brown-Bright-Green (BrBG)
Pink-Yellow-Green (PiYG)
Purple-Green (PRGn)
Purple-Orange (PuOr)
Red-Blue (RdBu)
Red-Grey (RdGy)
Red-Yellow-Blue (RdYlBu)
Red-Yellow-Green (RdYlGn)
Spectral (Spectral)

Example - the red-blue color scale to display your heatmap. But a reverse scale, which is Blue-Red colorscheme are more often used, so you might want to specify reversescale = TRUE

library(plotly)
fig <- plot_ly(z = volcano, type = "heatmap",
               colors = "RdBu", reversescale =T)
fig

Qualitative colorscale

Qualitative is useful when you want to disply categorical data. The available qualitative colorscales in plotly are:

Accent
Dark2
Paired
Pastel1
Pastel2
Set1
Set2
Set3

For more details, please take a look at https://colorbrewer2.org/#type=qualitative&scheme=Set1&n=3.

Scale

Sometimes users might found themselves in such pain when they try to adjust the range of colorscale in plotly, because it is not well-documented on the official website. For instance, in the volcano dataset, the temperature is usually considered to be greater than 0F. Therefore, it will be misleading if we plot it in blue color when temperature is greater than 0.

Fortunately, it is possible to adjust the range of your colorscale by specifying two arguments in plot_ly function.

zauto: if you specify this option to FALSE, then plotly won’t automatically adjust the colorscale in your heatmap.
zmax: the upper bound of your colorscale.
zmim: the lower bound of your colorscale.

# Expanding the range of the colorscale
fig <- plot_ly(z = volcano, type = "heatmap",
               colors = "RdBu", reversescale =T,
               zauto = FALSE, zmax = 200, zmin = 0)
fig

Borders and axis

Sometimes, it will be useful to add a border to your heatmap, especially when the boundary is not clear. A way to do it is to configure and customize the axis. Here’s an example of how to add borders to your heatmap:

ax <- list(
  zeroline = TRUE,
  showline = TRUE,
  mirror = "ticks",
  gridcolor = toRGB("gray50"),
  gridwidth = 2,
  zerolinecolor = toRGB("red"),
  zerolinewidth = 4,
  linecolor = toRGB("black"),
  linewidth = 6,
  titlefont=list(
  family = "Roman",
  size = 25
  )
)

ay <- list(
  zeroline = TRUE,
  showline = TRUE,
  mirror = "ticks",
  gridcolor = toRGB("gray50"),
  gridwidth = 2,
  zerolinecolor = toRGB("red"),
  zerolinewidth = 4,
  linecolor = toRGB("black"),
  linewidth = 6
)

fig <- plot_ly(z = volcano, type = "heatmap",
               colors = "RdBu", reversescale =T,
               zauto = FALSE, zmax = 200, zmin = 0) %>%
  layout(xaxis = ax, yaxis = ay)
fig

Subplot

subplot() is a very handy function to gather plotly objects.

fig_rb <- plot_ly(z = volcano, type = "heatmap",
                  colors = "RdBu", reversescale =T,
                  zauto = FALSE, zmax = 200, zmin = 0)
fig_gry <- plot_ly(z = volcano, type = "heatmap",
                   colors = "Greys", reversescale =T,
                   zauto = FALSE, zmax = 200, zmin = 0)
fig_grn <- plot_ly(z = volcano, type = "heatmap",
                   colors = "Greens", reversescale =T,
                   zauto = FALSE, zmax = 200, zmin = 0)
subplot(fig_rb, fig_gry, fig_grn)

R-Blog: ORCA - How to Save EPS Figures in R Plotly

2021-01-15T19:34:30+00:00

Save EPS Figures in R Plotly

It is not very common to save figures using EPS format.

One disadvantage of R Plotly is when you want to save figures from plotly, you have to use the “screenshot” buttom in RStudio, but the resulting figures is extremely large and high-quality.

In R, you can use the orca function provided by R Plotly. With the plotly R package, you can export graphs you create as static images in the .png, .jpg/.jpeg, .eps, .svg, and/or .pdf formats using Orca, an open source command line tool for generating static images of graphs created with Plotly’s graphing libraries.

Installation

As described in Orca, there are several ways to install orca in your machine

Method 1: Conda (Recommended)

$ conda install -c plotly plotly-orca

Method 2: npm (If you have Node.js installed)

$ npm install -g electron@6.1.4 orca

Method 3: Docker

$ docker pull quay.io/plotly/orca

Orca in RStudio

If you are using Rstudio, you might receive the following error message:

> plotly::orca_serve()
Error: The orca command-line utility is required for this functionality.

Please follow the installation instructions here -- https://github.com/plotly/orca#installation

That’s because RStudio doesn’t recognize the PATH environment variables for conda. As suggested by Github issue comment, you can fix this by telling RStudio where conda located.

Sys.setenv("PATH" = paste(Sys.getenv("PATH"), "[Your Conda /bin/ directory]", sep = .Platform$path.sep))

Now you are able to use orca in RStudio:

> Sys.setenv("PATH" = paste(Sys.getenv("PATH"), "/Users/xx/anaconda3/bin/", sep = .Platform$path.sep))
> plotly::orca_serve()
$port
[1] 5151

$process
PROCESS 'orca', running, pid 59108.

Export R Graphics as Static Images using `orca()`

For more details, please visit plotly official documentation for exporting static graphics in R. Here’s an example:

library(plotly)

if (!require("processx")) install.packages("processx")
fig <- plot_ly(z = ~volcano) %>% add_surface()
orca(fig, "surface-plot.svg")

R-Blog: Accessing list object in Rcpp

2021-01-12T19:34:30+00:00

Accessing list object in Rcpp using RcppArmadillo.

Create a list object in Rcpp

In Rcpp, you can create a list of object by using List::create() function. The name of element can be created using either Rcpp::Named() or _[].

// Create list L from vector v1, v2
List L = List::create(v1, v2);

// When giving names to elements
List L = List::create(Named("name1") = v1 , _["name2"] = v2);

Accessing the elements in list

You can easily access the element in a list by either using indices or character name.

NumericVector v1 = L[0];
NumericVector v2 = L["V1"];

However, notice that the objects in list are stored in Rcpp::Vector form, therefore, if you are using RcppArmadillo, you need to convert it to arma::mat :

arma::mat v1_mat = as<arma::mat>(L[0]);
arma::mat v2_mat = as<arma::mat>(L["V1"]);

Zikai Lin

Algorithms with Leetcode: Backtracking

Backtracking

78 Subsets

Breakdown

Conclusion

90. Subsets II

Problem Description

39 Combination Sum

Problem description

Breakdown

Python solution

40. Combination Sum II

Breakdown

Code

77. Combinations

46. Permutations

Breakdown

Solution:

Modified Squared Exponential Kernel and Basis Expansion

Squared Exponential Kernel

Modified Squared Exponential Kernel

Fast Basis Function Expansion based on Eigen Decomposition

Reference

R-Blog: Heatmap using R Plotly

Basics

Colors

Continuous color scale

Diverging colorscale

Qualitative colorscale

Scale

Borders and axis

Subplot

R-Blog: ORCA - How to Save EPS Figures in R Plotly

Save EPS Figures in R Plotly

Installation

Method 1: Conda (Recommended)

Method 2: npm (If you have Node.js installed)

Method 3: Docker

Orca in RStudio

Export R Graphics as Static Images using orca()

R-Blog: Accessing list object in Rcpp

Accessing list object in Rcpp using RcppArmadillo.

Create a list object in Rcpp

Accessing the elements in list

Export R Graphics as Static Images using `orca()`