diff --git a/.gitignore b/.gitignore index 8056204..223659c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,7 @@ *.db +.archiv .DS_Store +pyproject.toml +poetry.lock +.gp.md diff --git a/.stylua.toml b/.stylua.toml new file mode 100644 index 0000000..b8fb3ac --- /dev/null +++ b/.stylua.toml @@ -0,0 +1,11 @@ +column_width = 120 +line_endings = "Unix" +indent_type = "Spaces" +# TODO: Switch to 2 after merging +indent_width = 4 +quote_style = "AutoPreferDouble" +call_parentheses = "Always" +collapse_simple_statement = "Never" + +[sort_requires] +enabled = true diff --git a/README.md b/README.md index 7d3f285..0f374ce 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,38 @@ # Usage-Tracker.nvim > The plugin is in ⚠️ active development, and you can expect breaking changes in the future. -> Also the code is kind of 💩, refactoring is required. + + + -NeoVim plugin with which you can track your usage. -The examples speak for themselves. +NeoVim plugin with which you can track your usage along git project, branch and filetype. Logs are stored locally, so no sensitive data leaves your machine. -# Install +![Screenshot of the optional html output](screenshot.webp) +Use `:UsageTrackerHTML` to see your statistics. -Use your favourite package installer, there are no parameters at the moment. For example: +# Setup + +Use your favourite package manager, configuration parameters are optional. +- [vim-plug](https://github.com/junegunn/vim-plug): ```lua Plug 'gaborvecsei/usage-tracker.nvim' ``` -# Setup +- [lazy.nvim](https://github.com/folke/lazy.nvim) +```lua +{ + "gaborvecsei/usage-tracker.nvim", + config = function() + require("usage-tracker").setup({}) + end, +} +``` + +# Configuration + +The setup function can be used to configure parameters. Pass some or all of the possible parameters. ```lua require('usage-tracker').setup({ @@ -25,21 +42,21 @@ require('usage-tracker').setup({ inactivity_threshold_in_min = 5, inactivity_check_freq_in_sec = 5, verbose = 0, - telemetry_endpoint = "" -- you'll need to start the restapi for this feature + telemetry_endpoint = "" -- you'll need to start a REST API endpoint for this }) ``` -| Variable | Description | Type | Default | -| ------------------------------ | --------------------------------------------------------------------------------- | ---- | ------- | -| `keep_eventlog_days` | How much days of data should we keep in the event log after a cleanup | int | 14 | -| `cleanup_freq_days` | Frequency of the cleanup job for the event logs | int | 7 | -| `event_wait_period_in_sec` | Event logs are only recorded if this much seconds are elapsed while in the buffer | int | 5 | -| `inactivity_threshold_in_min` | If the cursor is not moving for this much time, the timer will be stopped | int | 5 | -| `inactivity_check_freq_in_sec` | How frequently check for inactivity | int | 1 | -| `verbose` | Debug messages are printed if it's `>0` | int | 1 | -| `telemetry_endpoint` | If defined data will be stored in a sqlite db via the restapi | str | '' | +| Parameter | Description | Type | Default | +| ------------------------------ | ---------------------------------------------------------------------------- | ---- | ------- | +| `keep_eventlog_days` | How much days of data should we keep in the event log after a cleanup | int | 14 | +| `cleanup_freq_days` | Frequency of the cleanup job for the event logs | int | 7 | +| `event_wait_period_in_sec` | Event logs are only recorded if this much seconds are elapsed in the buffer | int | 5 | +| `inactivity_threshold_in_min` | If the cursor is not moving for this number of minutes, the timer is stopped | int | 5 | +| `inactivity_check_freq_in_sec` | How frequently check for inactivity (seconds) | int | 1 | +| `verbose` | Debug messages are printed if it's `>0` | int | 1 | +| `telemetry_endpoint` | If defined data will be stored in a sqlite db via the restapi | str | '' | +| `json_file` | path of the text file which stores the usage data | str | cache folder | -(The variables are in the global space with the prefix `usagetracker_`) # Usage @@ -47,7 +64,7 @@ A timer starts when you enter a buffer and stops when you leave the buffer (or q Both normal and insert mode is counted. There is inactivity detection, which means that if you don't have any keys pressed down (normal, insert mode) then -the timer is stopped automatically. Please see the configuration to set your personal preference. +the timer is stopped automatically. ## Commands @@ -61,11 +78,13 @@ the timer is stopped automatically. Please see the configuration to set your per - `UsageTrackerShowDailyAggregationByFiletypes [filetypes]` - E.g.: `:UsageTrackerShowDailyAggregationByFiletypes lua markdown jsx` - `UsageTrackerShowDailyAggregationByProject [project_name]` -- `:UsageTrackerRemoveEntry ` +- `UsageTrackerRemoveEntry ` - This is a utility function with which you can remove a wrongly logged item from the json -- `:UsageTrackerClenup ` - -## Telemetry (separately storing data in a DB) +- `UsageTrackerCleanup ` +- `UsageTrackerHTML` + - This creates a html site in your cache folder which contains the data for visualization. Again, no data leaves your computer. + +## Optional telemetry endpoint (separately storing data in a DB) Usage data saved locally (in the json file) is cleaned up after the set days, but if you'd like to keep it longer in a separate SqliteDB, then this is why this feature exists. @@ -77,13 +96,14 @@ You can use it for custom analysis, just make sure the endpoint is live. ```console $ git clone https://github.com/gaborvecsei/usage-tracker.nvim.git $ cd usage-tracker.nvim/telemetry_api -$ docker-compose up -d +$ docker compose up -d ``` -If you'd like to use a different volume mount then change it in the `docker-compose.yml` file +Then you should define the parameter `telemetry_endpoint="http://:"` (if you did not change a thing the endpoint is `http://localhost:8000`) +parameter in the `setup({..., telemetry_endpoint="http://localhost:8000"})`. -Then you should define the `telemetry_endpoint="http://:"` (if you did not changed a thing the endpoint is `http://localhost:8000`) -parameter in the `setup({..., telemetry_endpoint="http://:"})`. +The database (sqlite file) is stored in `usage-tracker.nvim/telemetry_api/sqlite_db/database.db`. +If you'd like to change that, set a different volume mount in the `docker-compose.yml` file ## Examples @@ -110,7 +130,7 @@ Filepath Keystrokes Time (min) Pro /.config/nvim/init.vim 200 1.56 ``` -You can view the file-specific event (entry, exit) with **`:UsageTrackerShowVisitLog [filepath]`**. +You can view the file-specific events (entry, exit) with **`:UsageTrackerShowVisitLog [filepath]`**. Call the function when you are at the file you are interested in without any arguments or you can provide the filename as an argument. An event pair is only saved when more time elapsed than `event_wait_period_in_sec` seconds between the entry and the exit. Here is an example output: @@ -133,7 +153,7 @@ Daily usage in minutes 2023-07-05 | ################################################################################ | 333.1 ``` -The data is stored in a json file called `usage_data.json` in the neovim config folder (`vim.fn.stdpath("config") .. "/usage_data.json"`) +The data is stored in a json file called `usage_data.json` in the neovim config folder (`vim.fn.stdpath("config") .. "/usage_data.json"`). This can be configured with the parameter `json_file` in the setup. # Troubleshooting @@ -144,6 +164,7 @@ These are some of the issues I found when using the plugin. Bughunters are alway - Local data does not match up with the telemetry DB 100% - Some items in the visit logs are not "closed" - there is no exit time - For some visit logs the elapsed time is just too big (you can't code 25 hours in a day) +- I use `mini.start` and the plugin somehow measured a long activity on the starter window. ## Issues & Solutions diff --git a/doc/tags b/doc/tags new file mode 100644 index 0000000..cd3819a --- /dev/null +++ b/doc/tags @@ -0,0 +1,3 @@ +usage-tracker.nvim usage-tracker.nvim.txt /*usage-tracker.nvim* +usage-tracker-configuration usage-tracker.nvim.txt /*usage-tracker-configuration* +:UsageTracker usage-tracker.nvim.txt /*:UsageTracker* diff --git a/doc/usage-tracker.nvim.txt b/doc/usage-tracker.nvim.txt new file mode 100644 index 0000000..2d6f67b --- /dev/null +++ b/doc/usage-tracker.nvim.txt @@ -0,0 +1,99 @@ +*usage-tracker.nvim* Tack your neovim activity and show stats +=============================================================================== +USAGE-TRACKER.NVIM + +=============================================================================== +INTRODUCTION *ut-introduction* + +Usage-Tracker.nvim is a NeoVim plugin for tracking your activity across Git +projects, branches, and filetypes. All data is stored locally. + + +=============================================================================== +CONFIGURATION *usage-tracker-configuration* + +Configure the plugin using the setup function with the following parameters: + +Call `require('usage-tracker').setup({})` with `` being: + + > + keep_eventlog_days + Days to retain event log data. Default is 14. + + cleanup_freq_days + Frequency of event log cleanup in days. Default is 7. + + event_wait_period_in_sec + Seconds to wait before logging events. Default is 5. + + inactivity_threshold_in_min + Minutes of inactivity before stopping the timer. Default is 5. + + inactivity_check_freq_in_sec + Frequency of inactivity checks in seconds. Default is 5. + + verbose Set to 1 or more for debug messages. + + telemetry_endpoint + REST API endpoint for storing data in a database. + + json_file Path for the local JSON file storing usage data. Default is in + the cache folder. + < + +COMMANDS *:UsageTracker* + +The following commands are available: + + > + :UsageTrackerShowAgg [start_date] [end_date] + Show aggregated usage data. + + :UsageTrackerShowFilesLifetime + Show lifetime stats for files. + + :UsageTrackerShowVisitLog [filepath] + Show visit logs for a filepath or all logs by default. + + :UsageTrackerShowDailyAggregation + Show daily aggregated usage data. + + :UsageTrackerShowDailyAggregationByFiletypes [filetypes] + Show daily usage by filetypes. + + :UsageTrackerShowDailyAggregationByProject [project_name] + Show daily usage by project. + + :UsageTrackerRemoveEntry + Remove an entry from the usage log. + + :UsageTrackerCleanup + Clean up old usage data. + + :UsageTrackerHTML + Create a local HTML file with visualized data. + < + +For optional telemetry database setup, use the provided docker-compose config +and set the `telemetry_endpoint` parameter in the setup function. + +EXAMPLES *ut-examples* + + > + :UsageTrackerShowAgg filetype 2023-07-07 2023-07-08 + Shows summary of usage by filetype between specified dates. + + :UsageTrackerShowFilesLifetime + Displays the number of keystrokes and time spent in each file. + + :UsageTrackerShowVisitLog + Outputs entry and exit times, and duration for visits to files. + + :UsageTrackerShowDailyAggregationByFiletypes lua python markdown + Daily usage stats filtered by specified filetypes. + < + +=============================================================================== + + +vim:tw=78:ts=8:ft=help:norl: diff --git a/html/analyze.html b/html/analyze.html new file mode 100644 index 0000000..99839bf --- /dev/null +++ b/html/analyze.html @@ -0,0 +1,148 @@ + + + + + + Usage Data Visualization + + + +
+
+ +
+
+ +
+
+
+ + + + + + + + + +
+ +
to
+ +
+ + + + + +
+ + + + + + diff --git a/html/analyze.js b/html/analyze.js new file mode 100644 index 0000000..51d6ec1 --- /dev/null +++ b/html/analyze.js @@ -0,0 +1,542 @@ +// import Chart from "chart.js/auto"; + +var usageData; + +function addInitLoader() { + // Function to initialize the whole process + async function init() { + const chartData = processDataForChart(usageData, "filetype", "keystrokes"); + renderChart(chartData, "usageChart"); + const tData = processDataForTimeSeries(usageData, "filetype", "keystrokes"); + renderTimeSeries(tData, "timelineChart", "Number of keystrokes"); + } + + // Initialize the data and then populate the dropdown + window.addEventListener("DOMContentLoaded", () => { + init().then(() => { + populateGitProjectDropdown(usageData); + addButtonListener(); + addStartStopListener(); + }); + }); +} + +function addButtonListener() { + const gob = document.getElementById("goButton"); + gob.addEventListener("click", function () { + aggKey = document.getElementById("aggregationKey").value; + usageMeasure = document.getElementById("usageMeasure").value; + period = document.getElementById("period").value; + gitProject = document.getElementById("gitProject").value; + dateFrom = document.getElementById("dateFrom").value; + dateTo = document.getElementById("dateTo").value; + const { timeFrom, timeTo } = convertPeriod(period, dateFrom, dateTo); + + console.log("Aggregation Key:", aggKey); + console.log("Usage Measure:", usageMeasure); + console.log("Usage Data Length:", usageData); + console.log("Period:", period); + console.log("Time From:", timeFrom); + console.log("Time To:", timeTo); + console.log("gitProject", gitProject); + const chartData = processDataForChart( + usageData, + aggKey, + usageMeasure, + gitProject, + timeFrom, + timeTo, + ); + renderChart(chartData, "usageChart"); + const tData = processDataForTimeSeries( + usageData, + aggKey, + usageMeasure, + gitProject, + timeFrom, + timeTo, + ); + + const measureLabel = + usageMeasure === "keystrokes" ? "Keystrokes" : "Elapsed Time (min)"; + renderTimeSeries(tData, "timelineChart", measureLabel); + }); +} + +function addStartStopListener() { + const periodSelect = document.getElementById("period"); + const container = document.getElementById("startstopcontainer"); + + function toggleDateInputs() { + const shouldShow = periodSelect.value === "startend"; + container.style.display = shouldShow ? "flex" : "none"; + } + + periodSelect.addEventListener("change", toggleDateInputs); +} + +/** + * Converts a given period into a specific time range. + * @param {string} period - The period to convert (e.g., 'today', 'this_week'). + * @param {string} dateFrom - The starting date of the period in 'YYYY-MM-DD' format. + * @param {string} dateTo - The ending date of the period in 'YYYY-MM-DD' format. + */ +function convertPeriod(period, dateFrom = "", dateTo = "") { + var now = new Date(); + var timeFrom; + var timeTo; + + // Calculate timestamps based on selected period + switch (period) { + case "24hours": + timeFrom = new Date(now.getTime() - 24 * 60 * 60 * 1000); + timeTo = now; + break; + case "alltime": + timeFrom = new Date(0); + timeTo = now; + break; + case "yesterday": + timeFrom = new Date(now.getFullYear(), now.getMonth(), now.getDate() - 1); + timeTo = new Date(timeFrom.getTime()); + timeTo.setHours(23, 59, 59, 999); + break; + case "today": + timeFrom = new Date(now.setHours(0, 0, 0, 0)); + timeTo = new Date(now.setHours(23, 59, 59, 999)); + break; + case "startend": + if (dateFrom) { + timeFrom = new Date(dateFrom); + timeFrom.setHours(0, 0, 0, 0); + } else { + timeFrom = new Date(0); // January 1, 1970 UTC + } + if (dateTo) { + timeTo = new Date(dateTo); + timeTo.setHours(23, 59, 59, 999); + } else { + timeTo = new Date(8640000000000000); // Set to far future date + } + break; + default: + throw new Error("Invalid period selected"); + } + + // console.log(timeFrom.toISOString()); + // console.log(timeTo.toISOString()); + // Convert Date objects to integer timestamps + timeFrom = Math.floor(timeFrom.getTime() / 1000); + timeTo = Math.floor(timeTo.getTime() / 1000); + return { timeFrom, timeTo }; +} + +/** + * Populates the Git project dropdown with unique project names from usage data. + * @param {Object} myusageData - The object containing usage data including git projects. + */ +function populateGitProjectDropdown(myusageData) { + const gitProjectSelect = document.getElementById("gitProject"); + if (!gitProjectSelect) return; + + // Extract unique git project entries + const extractUniqueGitProjects = (myusageData) => { + const uniqueProjects = new Set(); + const dataEntries = Object.values(myusageData.data); + + dataEntries.forEach((entry) => { + if (entry.git_project_name) { + uniqueProjects.add(entry.git_project_name); + } + }); + + return Array.from(uniqueProjects); + }; + + const uniqueGitProjects = extractUniqueGitProjects(myusageData); + + // Create dropdown options + uniqueGitProjects.forEach((gitProject) => { + const option = document.createElement("option"); + option.value = gitProject; + option.textContent = gitProject; + gitProjectSelect.appendChild(option); + }); +} + +/** + * Processes the raw usage data to create datasets suitable for chart visualization based on a specified aggregation key and usage measure. + * @param {Object} inusageData - An object containing the usage data for neovim editor sessions. from usage_data table in init.lua + * @param {string} aggKey - The key to aggregate data by. Accepted keys are 'git_branch', 'git_project_name', or 'filetype'. + * @param {string} usageMeasure - The measure of usage to process. Accepted measures are 'keystrokes' or 'elapsed_time_sec'. + * @param {String} gitProject The name of the git project to filter the data by. + * @param {Date} timeFrom The start date-time from which to filter the usage data. + * @param {Date} timeTo The end date-time until which to filter the usage data. + * @returns {Map} A map where each key is a label generated from the aggKey and each value is the corresponding dataset for the chart. + * @throws Will throw an error if the usageMeasure or aggKey is invalid. + */ +function processDataForChart( + inusageData, + aggKey, + usageMeasure, + gitProject = "all", + timeFrom = -9999999999999, + timeTo = 9999999999999, +) { + if (!["keystrokes", "elapsed_time_sec"].includes(usageMeasure)) { + throw new Error(`Invalid usage measure: ${usageMeasure}`); + } + if (!["git_branch", "git_project_name", "filetype"].includes(aggKey)) { + throw new Error(`Invalid aggregation key: ${aggKey}`); + } + + const measureLabel = + usageMeasure === "keystrokes" ? "Keystrokes" : "Elapsed Time (min)"; + + // Initialize the output object + var chartData = { + labels: [], + datasets: [ + { + label: measureLabel, + data: [], + }, + ], + }; + + var label; + // Loop over very filepath (key in usageData.data) + for (const filepath in inusageData.data) { + const fileData = inusageData.data[filepath]; + if (fileData.git_project_name !== gitProject && gitProject !== "all") { + continue; + } + switch (aggKey) { + case "git_branch": + label = `${fileData.git_project_name} (${fileData.git_branch})`; + break; + case "git_project_name": + label = `${fileData.git_project_name}`; + break; + case "filetype": + label = fileData[aggKey]; + break; + default: + throw new Error(`Invalid aggregation key: ${aggKey}`); + } + // sum over the usage measure (keystrokes or elapsed_time_sec) + let measureSum = fileData.visit_log.reduce( + (sum, log) => + log.entry >= timeFrom && log.entry <= timeTo + ? sum + log[usageMeasure] + : sum, + 0, + ); + + // convert to minutes for better readability + if (usageMeasure == "elapsed_time_sec") { + measureSum /= 60; + measureSum = Math.round(measureSum); + } + + // Initialize if label does not exist + if (!chartData.labels.includes(label)) { + chartData.labels.push(label); + chartData.datasets[0].data.push(0); // Initialize sum for this label + } + // Add the measure to the respective label + const labelIndex = chartData.labels.indexOf(label); + chartData.datasets[0].data[labelIndex] += measureSum; + } + + // Some aggregation keys are empty (like the git project) + chartData.labels.forEach((label, index, labels) => { + if (label === "") { + labels[index] = ""; + } + }); + + // Order by measureSum in descending order + const orderedData = chartData.labels + .map((label, index) => ({ label, sum: chartData.datasets[0].data[index] })) + .sort((a, b) => b.sum - a.sum); + + // Reassign sorted data to chartData + chartData.labels = orderedData.map((item) => item.label); + chartData.datasets[0].data = orderedData.map((item) => item.sum); + + // Aggregating all elements beyond a certain threshold into an 'Other' category + const aggregationThreshold = 5; // Define your own threshold here + if (chartData.labels.length > aggregationThreshold) { + const aggregatedData = { + label: "Other", + sum: chartData.datasets[0].data + .slice(aggregationThreshold) + .reduce((sum, value) => sum + value, 0), + }; + + // Set labels and data for the top items and the 'Other' category + chartData.labels = chartData.labels + .slice(0, aggregationThreshold) + .concat(aggregatedData.label); + chartData.datasets[0].data = chartData.datasets[0].data + .slice(0, aggregationThreshold) + .concat(aggregatedData.sum); + } + return chartData; +} + +/** + * Processes the raw usage data to create datasets suitable for chart visualization based on a specified aggregation key and usage measure. + * @param {Object} inusageData - An object containing the usage data for neovim editor sessions. from usage_data table in init.lua + * @param {string} aggKey - The key to aggregate data by. Accepted keys are 'git_branch', 'git_project_name', or 'filetype'. + * @param {string} usageMeasure - The measure of usage to process. Accepted measures are 'keystrokes' or 'elapsed_time_sec'. + * @param {String} gitProject The name of the git project to filter the data by. + * @param {Date} timeFrom The start date-time from which to filter the usage data. + * @param {Date} timeTo The end date-time until which to filter the usage data. + * @returns {Map} A map where each key is a label generated from the aggKey and each value is the corresponding dataset for the chart. + * @throws Will throw an error if the usageMeasure or aggKey is invalid. + */ +function processDataForTimeSeries( + inusageData, + aggKey, + usageMeasure, + gitProject = "all", + timeFrom = -9999999999999, + timeTo = 9999999999999, +) { + if (!["keystrokes", "elapsed_time_sec"].includes(usageMeasure)) { + throw new Error(`Invalid usage measure: ${usageMeasure}`); + } + if (!["git_branch", "git_project_name", "filetype"].includes(aggKey)) { + throw new Error(`Invalid aggregation key: ${aggKey}`); + } + + // First Generate a dictionary where the keys are dates and the values are dictionaries + // holding the sum of usageMeasure per aggregation key + var datesAndAggKeyWithUsage = {}; + + var label; + // Loop over very filepath (key in usageData.data) + for (const filepath in inusageData.data) { + const fileData = inusageData.data[filepath]; + if (fileData.git_project_name !== gitProject && gitProject !== "all") { + continue; + } + switch (aggKey) { + case "git_branch": + label = `${fileData.git_project_name} (${fileData.git_branch})`; + break; + case "git_project_name": + label = `${fileData.git_project_name}`; + break; + case "filetype": + label = fileData[aggKey]; + break; + default: + throw new Error(`Invalid aggregation key: ${aggKey}`); + } + + // Only include data within the specified time range and for the specified git project + for (const visit of fileData.visit_log) { + if (visit.entry >= timeFrom && visit.entry <= timeTo) { + // Convert the entry timestamp to ISO date format + const date = new Date(visit.entry * 1000).toISOString().split("T")[0]; + // If the date is not in the dictionary, initialize it + if (!datesAndAggKeyWithUsage[date]) { + datesAndAggKeyWithUsage[date] = {}; + } + // If the usage measure key is not in the date object, initialize it + if (!datesAndAggKeyWithUsage[date][label]) { + datesAndAggKeyWithUsage[date][label] = 0; + } + // Add the usage measure to the date's sum + datesAndAggKeyWithUsage[date][label] += visit[usageMeasure]; + } + } + } + + // The datasets entries represent the different aggregation keys + // The labels entries represent the days in iso format + var timeData = { + labels: [], + datasets: [], + }; + // Sort the dates and use them as labels for the chart + timeData.labels = Object.keys(datesAndAggKeyWithUsage).sort(); + + // Create datasets for each unique aggregation key + const uniqueKeys = new Set(); + for (const date in datesAndAggKeyWithUsage) { + for (const key in datesAndAggKeyWithUsage[date]) { + uniqueKeys.add(key); + } + } + + uniqueKeys.forEach((key) => { + const dataset = { + label: key, + data: [], + }; + timeData.labels.forEach((date) => { + var multiplyer = 1; + if (usageMeasure === "elapsed_time_sec") { + multiplyer = 60; + } + dataset.data.push(datesAndAggKeyWithUsage[date][key] / multiplyer || 0); + }); + timeData.datasets.push(dataset); + }); + + return timeData; +} + +/** + * Renders a chart with the given data. + * @param {Object} chartData - The data to be used in the chart. + * @param {string} elementId - The ID of the HTML element where the chart will be rendered. + */ +function renderChart(chartData, elementId) { + // Destroy any existing chart + const chart = Chart.getChart(elementId); + if (chart) { + chart.destroy(); + } + const ctx = document.getElementById(elementId).getContext("2d"); + new Chart(ctx, { + type: "bar", + data: chartData, + options: { + indexAxis: "y", + scales: { + x: { + beginAtZero: true, + ticks: { + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + }, + }, + }, + y: { + ticks: { + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + color: "black", + }, + }, + }, + }, + plugins: { + legend: { + display: false, + }, + title: { + display: true, + text: chartData.datasets[0].label, + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + }, + }, + }, + }, + }); +} + +/** + * Renders a time series stacked bar chart using Chart.js. + * @param {Object} timeData - The processed data with labels and datasets for the chart. + * @param {String} canvasId - The ID of the canvas element where the chart will be rendered. + * @param {String} title - The title of the chart. + */ +function renderTimeSeries(timeData, canvasId, title = "Usage") { + const chart = Chart.getChart(canvasId); + if (chart) { + chart.destroy(); + } + const ctx = document.getElementById(canvasId).getContext("2d"); + new Chart(ctx, { + type: "bar", + data: { + labels: timeData.labels, + datasets: timeData.datasets.map((dataset) => ({ + ...dataset, + backgroundColor: getRandomColor(), // Utility function needed to get random colors for the bars + stack: "Stack 0", // All datasets are part of the same stack + })), + }, + options: { + plugins: { + legend: { + display: true, + position: "bottom", + labels: { + color: "black", + font: { + size: 10, + family: "Ubuntu, Roboto, sans-serif", + }, + }, + }, + title: { + display: true, + text: title, + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + }, + }, + }, + scales: { + x: { + stacked: true, // Stacks the bars on the x-axis + ticks: { + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + color: "black", + }, + }, + }, + y: { + stacked: true, // Stacks the bars on the y-axis + ticks: { + color: "black", + font: { + size: 18, + family: "Ubuntu, Roboto, sans-serif", + color: "black", + }, + }, + }, + }, + }, + }); +} + +/** + * Generates a random color in the RGBA format. + * Note: In a production environment, it's often better to use a predefined set of colors. + * @returns {String} A string representing an RGBA color. + */ +function getRandomColor() { + const r = Math.floor(Math.random() * 255); + const g = Math.floor(Math.random() * 255); + const b = Math.floor(Math.random() * 255); + return `rgba(${r}, ${g}, ${b}, 0.7)`; +} +// This is needed for testing with jest +if (typeof module !== "undefined" && module.exports) { + module.exports = { + processDataForChart, + processDataForTimeSeries, + convertPeriod, + }; +} diff --git a/html/package.json b/html/package.json new file mode 100644 index 0000000..7fb1c4c --- /dev/null +++ b/html/package.json @@ -0,0 +1,19 @@ +{ + "name": "usage-tracker.nvim", + "version": "1.0.0", + "type": "module", + "main": "analyze.js", + "jest": { + "transform": {} + }, + "scripts": { + "test": "node --experimental-vm-modules node_modules/jest/bin/jest.js", + "build": "npx webpack --config webpack.conf.cjs" + }, + "devDependencies": { + "jest": "^27.5.1", + "chart.js": "^4.0.0", + "webpack": "^5.72.0", + "webpack-cli": "*" + } +} diff --git a/html/test.js b/html/test.js new file mode 100644 index 0000000..9cd2f2f --- /dev/null +++ b/html/test.js @@ -0,0 +1,214 @@ +// run with +// npm install jest --global +// jest + +const { + processDataForChart, + convertPeriod, + processDataForTimeSeries, +} = require("./analyze"); + +describe("convertPeriod", () => { + // Set up a mock date + const mockNow = new Date(2023, 3, 10); // April 10th, 2023 + jest.useFakeTimers().setSystemTime(mockNow); + + test('returns the correct range for "24hours"', () => { + const { timeFrom, timeTo } = convertPeriod("24hours"); + const expectedTimeFrom = Math.floor( + new Date(mockNow.getTime() - 24 * 60 * 60 * 1000).getTime() / 1000, + ); + expect(timeFrom).toBe(expectedTimeFrom); + expect(timeTo).toBe(Math.floor(mockNow.getTime() / 1000)); + }); + + test('returns the correct range for "alltime"', () => { + const { timeFrom, timeTo } = convertPeriod("alltime"); + expect(timeFrom).toBe(Math.floor(new Date(0).getTime() / 1000)); + expect(timeTo).toBe(Math.floor(mockNow.getTime() / 1000)); + }); + + test("throws an error for invalid periods", () => { + expect(() => { + convertPeriod("invalidPeriod"); + }).toThrow("Invalid period selected"); + }); + + afterAll(() => { + jest.useRealTimers(); + }); +}); + +describe("Test processDataForChart", () => { + let testusageData; + + beforeEach(() => { + testusageData = { + data: { + "/path/to/file1.lua": { + git_project_name: "example-project-1", + git_branch: "main", + filetype: "python", + visit_log: [ + { + entry: 1669990000, + exit: 1670000000, + elapsed_time_sec: 6000, + keystrokes: 100, + }, + ], + }, + "/path/to/file2.py": { + git_project_name: "example-project-2", + git_branch: "feature", + filetype: "python", + visit_log: [ + { + entry: 1669990000, + exit: 1670000000, + elapsed_time_sec: 4000, + keystrokes: 50, + }, + ], + }, + }, + }; + }); + + it("should aggregate data by git project name with keystrokes measure", () => { + const expectedChartData = { + labels: ["example-project-1", "example-project-2"], + datasets: [ + { + label: "Keystrokes", + data: [100, 50], + }, + ], + }; + + const chartData = processDataForChart( + testusageData, + "git_project_name", + "keystrokes", + ); + expect(chartData).toEqual(expectedChartData); + }); + + it("should aggregate data by git branch with elapsed time measure", () => { + const expectedChartData = { + labels: ["example-project-1 (main)", "example-project-2 (feature)"], + datasets: [ + { + label: "Elapsed Time (min)", + data: [100, 67], // Rounded elapsed time in minutes + }, + ], + }; + + const chartData = processDataForChart( + testusageData, + "git_branch", + "elapsed_time_sec", + ); + expect(chartData).toEqual(expectedChartData); + }); + + it("should aggregate data by filetype with elapsed time measure", () => { + const expectedChartData = { + labels: ["python"], + datasets: [ + { + label: "Elapsed Time (min)", + data: [167], // Rounded elapsed time in minutes from all python files + }, + ], + }; + + const chartData = processDataForChart( + testusageData, + "filetype", + "elapsed_time_sec", + ); + expect(chartData).toEqual(expectedChartData); + }); + + it("should throw an error for invalid usage measure", () => { + expect(() => { + processDataForChart(testusageData, "git_project_name", "invalid_measure"); + }).toThrowError("Invalid usage measure: invalid_measure"); + }); + + it("should throw an error for invalid aggregation key", () => { + expect(() => { + processDataForChart(testusageData, "invalid_key", "keystrokes"); + }).toThrowError("Invalid aggregation key: invalid_key"); + }); +}); + +describe("processDataForTimeSeries", () => { + test("should process data correctly with valid input", () => { + const usageData = { + data: { + "/path/to/file1.js": { + git_project_name: "project1", + git_branch: "main", + filetype: "javascript", + visit_log: [ + { entry: 1610000000, keystrokes: 100, elapsed_time_sec: 60 }, + { entry: 1610000600, keystrokes: 50, elapsed_time_sec: 30 }, + ], + }, + "/path/to/file2.py": { + git_project_name: "project2", + git_branch: "dev", + filetype: "python", + visit_log: [ + { entry: 1610001200, keystrokes: 200, elapsed_time_sec: 120 }, + { entry: 1610001800, keystrokes: 100, elapsed_time_sec: 60 }, + ], + }, + }, + }; + const expectedTimeData = { + labels: ["2021-01-07"], + datasets: [ + { + label: "project1 (main)", + data: [150], + }, + { + label: "project2 (dev)", + data: [300], + }, + ], + }; + const timeData = processDataForTimeSeries( + usageData, + "git_branch", + "keystrokes", + "all", + new Date("2021-01-07T00:00:00Z").getTime() / 1000, + new Date("2021-01-08T00:00:00Z").getTime() / 1000, + ); + + expect(timeData).toEqual(expectedTimeData); + }); + + test("should throw error for invalid usageMeasure", () => { + const usageData = { + /* ... */ + }; // provide minimal mock data + expect(() => { + processDataForTimeSeries(usageData, "git_branch", "invalidMeasure"); + }).toThrow(Error); + }); + + test("should throw error for invalid aggKey", () => { + const usageData = { + /* ... */ + }; // provide minimal mock data + expect(() => { + processDataForTimeSeries(usageData, "invalidKey", "keystrokes"); + }).toThrow(Error); + }); +}); diff --git a/lua/usage-tracker/agg.lua b/lua/usage-tracker/agg.lua index 2a07f38..c094f7d 100644 --- a/lua/usage-tracker/agg.lua +++ b/lua/usage-tracker/agg.lua @@ -2,7 +2,6 @@ local utils = require("usage-tracker.utils") local M = {} - --- Creates a lifetime aggregation of the visit logs for each file present in the usage data ---@param usage_data table ---@return table @@ -31,7 +30,7 @@ function M.lifetime_aggregation_of_visit_logs(usage_data) elapsed_time_in_min = total_elapsed_time_min, elapsed_time_in_hour = total_elapsed_time_hour, } - result[#result + 1] = result_item + table.insert(result, result_item) end return result @@ -40,26 +39,34 @@ end --- Creates a daily aggregation of the visit logs --- Example for the daily aggregation: --- {{day: 2022-01-02, time_in_sec: 2345, keystrokes: 1234}, {day: 2022-01-03, time_in_sec: 2345, keystrokes: 1234}, ...} ----@param filetypes string[] Filetypes which we would like to include, if empty then we don't filter for any filetypes and everything is included ----@param project_name string Project name which we would like to include, if empty then we don't filter for any project and everything is included +---@param filetypes string[]? Filetypes which we would like to include, if empty then we don't filter for any filetypes and everything is included +---@param project_name string? Project name which we would like to include, if empty then we don't filter for any project and everything is included ---@return table function M.create_daily_usage_aggregation(usage_data, filetypes, project_name) local result = {} for filepath, file_data in pairs(usage_data.data) do - if (filetypes == nil or utils.list_contains(filetypes, file_data.filetype)) and (project_name == nil or project_name == file_data.git_project_name) then + if + (filetypes == nil or utils.list_contains(filetypes, file_data.filetype)) + and (project_name == nil or project_name == file_data.git_project_name) + then local visit_log = file_data.visit_log for _, row_data in ipairs(visit_log) do -- We'll use the entry time as the key for the result table local entry_day_date = utils.timestamp_to_date(row_data.entry, true) local entry_day_date_str = os.date("%Y-%m-%d", row_data.entry) - local exit_day_date = utils.timestamp_to_date(row_data.exit, true) + -- local exit_day_date = utils.timestamp_to_date(row_data.exit, true) local exit_day_date_str = os.date("%Y-%m-%d", row_data.exit) if entry_day_date_str ~= exit_day_date_str then utils.verbose_print( - "Entry and exit date are different, we'll use the entry date during the aggregation. Entry: " .. - entry_day_date_str .. ", exit: " .. exit_day_date_str .. ", filepath: " .. filepath) + "Entry and exit date are different, we'll use the entry date during the aggregation. Entry: " + .. entry_day_date_str + .. ", exit: " + .. exit_day_date_str + .. ", filepath: " + .. filepath + ) end local time_in_sec = row_data.elapsed_time_sec @@ -116,17 +123,16 @@ function M.create_daily_usage_aggregation(usage_data, filetypes, project_name) current_day_timestamp = utils.increment_timestamp_by_days(current_day_timestamp, 1) end - -- Flatten the table and then order it based on the date local result_table = {} for day_date_str, data in pairs(result) do - result_table[#result_table + 1] = { + table.insert(result_table, { day_str = day_date_str, day_timestamp = utils.date_to_timestamp(data.day), time_in_sec = data.time_in_sec, time_in_min = math.floor(data.time_in_sec / 60 * 100) / 100, - keystrokes = data.keystrokes - } + keystrokes = data.keystrokes, + }) end table.sort(result_table, function(a, b) @@ -155,7 +161,7 @@ function M.aggregate(usage_data, key, start_date_timestamp, end_date_timestamp) if key == "filetype" then agg_field_value = file_data.filetype elseif key == "project" then - agg_field_value = file_data.git_project_name + agg_field_value = file_data.git_project_name .. "." .. file_data.git_branch elseif key == "filepath" then agg_field_value = filepath else @@ -186,11 +192,11 @@ function M.aggregate(usage_data, key, start_date_timestamp, end_date_timestamp) -- Flatten the table and then order it based on the elapsed_time_sec local result_table = {} for agg_field_value, data in pairs(result) do - result_table[#result_table + 1] = { + table.insert(result_table, { name = agg_field_value, time_in_sec = data.time_in_sec, - keystrokes = data.keystrokes - } + keystrokes = data.keystrokes, + }) end return result_table diff --git a/lua/usage-tracker/config.lua b/lua/usage-tracker/config.lua new file mode 100644 index 0000000..8254f0e --- /dev/null +++ b/lua/usage-tracker/config.lua @@ -0,0 +1,35 @@ +local M = {} + +---@class Config +---@field keep_eventlog_days? integer +---@field cleanup_freq_days integer +---@field event_wait_period_in_sec integer +---@field inactivity_threshold_in_min integer +---@field inactivity_check_freq_in_sec integer +---@field verbose integer +---@field telemetry_endpoint string +---@field json_file string + +---@type Config +M.config = { + keep_eventlog_days = 4, + cleanup_freq_days = 7, + event_wait_period_in_sec = 5, + inactivity_threshold_in_min = 2, + inactivity_check_freq_in_sec = 1, + verbose = 0, + telemetry_endpoint = "", + --TODO: The json file should be replaced by a csv, so that append write is faster + -- config path for backward compatibility + json_file = vim.fn.stdpath("config") .. "/usage_data.json", +} + +function M.setup_config(opts) + for opt, _ in pairs(M.config) do + if opts[opt] ~= nil then + M.config[opt] = opts[opt] + end + end +end + +return M diff --git a/lua/usage-tracker/draw.lua b/lua/usage-tracker/draw.lua index 7cfc717..c841908 100644 --- a/lua/usage-tracker/draw.lua +++ b/lua/usage-tracker/draw.lua @@ -3,15 +3,17 @@ local M = {} --- Draws (on the message bar) a vertical barchart for the aggregated daily data -- data looks like this: {{name: 2022-02-30, value: 235.67}, ...} ---@param data table The data should have a name and a value field ----@param max_chars integer The maximum number of characters to use for the bar ----@param title string The title of the chart ----@param sort boolean Whether to sort the data by value ----@param mnl integer The maximum number of characters to use for the name (y axis) +---@param max_chars integer? The maximum number of characters to use for the bar +---@param title string? The title of the chart +---@param sort boolean? Whether to sort the data by value +---@param mnl integer? The maximum number of characters to use for the name (y axis) function M.vertical_barchart(data, max_chars, title, sort, mnl) max_chars = max_chars or 60 title = title or "" sort = sort or false mnl = mnl or 30 + ---@type table> + local chunks = {} if sort then table.sort(data, function(a, b) @@ -22,7 +24,9 @@ function M.vertical_barchart(data, max_chars, title, sort, mnl) local max_value = 0 local max_name_length = 0 - print(title .. "\n" .. string.rep("-", #title) .. "\n") + -- print(title .. "\n" .. string.rep("-", #title) .. "\n") + table.insert(chunks, { title .. "\n" .. string.rep("-", #title) .. "\n", "@markup.heading.5.markdown" }) + table.insert(chunks, { " ", "" }) for _, item in ipairs(data) do max_value = math.max(max_value, item.value) @@ -34,12 +38,14 @@ function M.vertical_barchart(data, max_chars, title, sort, mnl) for _, item in ipairs(data) do local bar_length = math.floor((item.value / max_value) * max_chars) - local bar = string.rep("#", bar_length) + local bar = string.rep("▇", bar_length) local value_string = string.format("%-" .. max_chars .. "s", tostring(item.value)) local name_string = string.format("%-" .. max_name_length .. "s", item.name) - local line = name_string .. " | " .. bar .. " | " .. value_string - print(line) + table.insert(chunks, { name_string .. " | ", "Boolean" }) + table.insert(chunks, { bar, "" }) + table.insert(chunks, { " | " .. value_string .. " \n ", "@comment.info" }) end + vim.api.nvim_echo(chunks, false, {}) end --- Prints the results in a table format to the messages @@ -52,6 +58,8 @@ end function M.print_table_format(headers, data, field_names) -- Calculate the maximum length needed for each column local maxLens = {} + ---@type table + local chunks = {} for i, header in ipairs(headers) do local field_name = field_names[i] maxLens[field_name] = #header @@ -75,8 +83,10 @@ function M.print_table_format(headers, data, field_names) separator = separator .. string.rep("-", l) .. " " end - print(string.format(headerFormat, unpack(headers))) - print(separator) + -- print(string.format(headerFormat, unpack(headers))) + -- print(separator) + table.insert(chunks, { string.format(headerFormat, unpack(headers)) .. "\n", "@markup.heading.5.markdown" }) + table.insert(chunks, { separator .. "\n", "Boolean" }) for _, rowData in ipairs(data) do local rowFormat = "" @@ -92,8 +102,10 @@ function M.print_table_format(headers, data, field_names) for i, field_name in ipairs(field_names) do rowValues[i] = rowData[field_name] end - print(string.format(rowFormat, unpack(rowValues))) + -- print(string.format(rowFormat, unpack(rowValues))) + table.insert(chunks, { string.format(rowFormat, unpack(rowValues)) .. "\n", "" }) end + vim.api.nvim_echo(chunks, false, {}) end return M diff --git a/lua/usage-tracker/html.lua b/lua/usage-tracker/html.lua new file mode 100644 index 0000000..44a7556 --- /dev/null +++ b/lua/usage-tracker/html.lua @@ -0,0 +1,107 @@ +local M = {} + +-- TODO: Test results against the neovim interface + +--- Get the path of the current script +---@return string path +local function get_script_path() + local str = debug.getinfo(2, "S").source:sub(2) + local win = package.config:sub(1, 1) == "\\" + local path_sep = "/" + if win then + str = str:gsub("/", "\\") + path_sep = "\\" + end + return str:match("(.*" .. path_sep .. ")") +end + +--- Read the contents of text file to a lua string +---@param file_path string path of file to be read +local function read_file(file_path) + local file = io.open(file_path, "r") + if file then + local content = file:read("*all") + file:close() + return content + else + return nil, "Could not read file" + end +end + +--- (Over-)write a string to a text file +---@param file_path string file path +---@param content string string to be written to the file +local function write_file(file_path, content) + local file = io.open(file_path, "w") + if file then + file:write(content) + file:close() + else + error("Could not write to file" .. file_path) + end +end + +--- Append a string to a text file +---@param file_path string file path +---@param content string string to be written to the file +local function append_file(file_path, content) + local file = io.open(file_path, "a") + if file then + file:write(content) + file:close() + else + error("Could not append to file" .. file_path) + end +end + +--- open a browser with a specific webpage +---@param file_path string file or url to be opened with the browser +local function open_browser(file_path) + local open_command + if vim.fn.has("mac") == 1 then + open_command = "open" + elseif vim.fn.has("unix") == 1 then + open_command = "xdg-open" + else + -- TODO: Windows support is not tested + open_command = "start" -- For Windows + end + vim.fn.jobstart({ open_command, file_path }, { detach = true }) +end + +--- Copy contents from one text file to the other +---@param source string source path +---@param target string target path +local function copy_file(source, target) + local bundle, bundle_err = read_file(source) + if not bundle then + error("Error reading js bundle file: " .. bundle_err) + end + write_file(target, bundle) +end + +--- Create a html file which contains the data and links to javascript code for visualization +function M.create_html_stats() + local json_file_path = require("usage-tracker.config").config.json_file + local html_template_path = get_script_path() .. "../../html/analyze.html" + + local json_data, json_err = read_file(json_file_path) + if not json_data then + error("Error reading JSON file: " .. json_err) + end + copy_file(get_script_path() .. "../../html/analyze.js", vim.fn.stdpath("cache") .. "/analyze.js") + + local html_template, html_err = read_file(html_template_path) + if not html_template then + error("Error reading HTML template file: " .. html_err) + end + + local html_content = html_template:gsub("{{ json_data_here }}", json_data) + -- html_content = html_content:gsub("{{ analyzejs }}", js_functions) + + local output_file_path = vim.fn.stdpath("cache") .. "/usage-tracker.html" + write_file(output_file_path, html_content) + open_browser(output_file_path) +end + +return M diff --git a/lua/usage-tracker/init.lua b/lua/usage-tracker/init.lua index a2750b7..15ed8e0 100644 --- a/lua/usage-tracker/init.lua +++ b/lua/usage-tracker/init.lua @@ -1,40 +1,54 @@ -local curl = require('plenary.curl') -local utils = require("usage-tracker.utils") -local draw = require("usage-tracker.draw") local agg = require("usage-tracker.agg") +local curl = require("plenary.curl") +local draw = require("usage-tracker.draw") +local utils = require("usage-tracker.utils") +-- Configuration to be used here and in the other files local M = {} +M.config = require("usage-tracker.config").config --- Global variables +-- Global state variables of the usage tracker --- We'll use this object for storing the data ----@type table -local usage_data = { last_cleanup = os.time(), data = {} } +---@class VisitLogEntry +---@field entry number|nil +---@field exit number|nil +---@field elapsed_time_sec number +---@field keystrokes number --- Use the Neovim config file path ----@type string -local jsonFilePath = vim.fn.stdpath("config") .. "/usage_data.json" +---@class UsagedataPerFilepath +---@field git_project_name string +---@field git_branch string +---@field filetype string +---@field visit_log table + +---@class UsageData +---@field last_cleanup number +---@field data table --- Variable to keep track of the last activity time - needed for inactivity "detection" +--- Storage for all usage data +---@type UsageData +local usage_data = { last_cleanup = os.time(), data = {} } + +--- Flag to track inactivity status (true for inactive, false for active) ---@type boolean local is_inactive = false ---@type number local last_activity_timestamp = os.time() --- Variable to keep track of the current buffer --- Mostly needed as we cannot use the vim.api.nvim_buf_get_name and vim.api.nvim_get_current_buf functions --- in the vim event loops (which is needed for the inactivity detection) +--- Variable to keep track of the current buffer +--- Mostly needed as we cannot use the vim.api.nvim_buf_get_name and vim.api.nvim_get_current_buf functions +--- in the vim event loops (which is needed for the inactivity detection) ---@type number local current_bufnr = nil + ---@type string local current_buffer_filepath = nil - --- Save the timers to the JSON file local function save_usage_data() - local encodedTimers = vim.json.encode(usage_data) - local file = io.open(jsonFilePath, "w") + local encodedTimers = vim.fn.json_encode(usage_data) + local file = io.open(M.config.json_file, "w") if file then file:write(encodedTimers) file:close() @@ -43,47 +57,63 @@ end --- Load the timers from the JSON file local function load_usage_data() - local file = io.open(jsonFilePath, "r") + local file = io.open(M.config.json_file, "r") if file then local encodedTimers = file:read("*all") file:close() - usage_data = vim.json.decode(encodedTimers) + usage_data = vim.json.decode(encodedTimers) or {} end end --- Send data to the restapi -local function send_data_to_restapi(filepath, entry_timestamp, exit_timestamp, keystrokes, filetype, git_project_name) +---@param filepath string +---@param entry_timestamp number +---@param exit_timestamp number +---@param keystrokes number +---@param filetype string +---@param git_project_name string +---@param git_branch string +local function send_data_to_restapi( + filepath, + entry_timestamp, + exit_timestamp, + keystrokes, + filetype, + git_project_name, + git_branch +) + if not M.config.telemetry_endpoint or M.config.telemetry_endpoint == "" then + return + end + local json = { entry = entry_timestamp, exit = exit_timestamp, keystrokes = keystrokes, filepath = filepath, filetype = filetype, - projectname = git_project_name, + git_project_name = git_project_name, + git_branch = git_branch, } - local telemetry_endpoint = vim.g.usagetracker_telemetry_endpoint - if telemetry_endpoint and telemetry_endpoint ~= "" then - local res = curl.post(telemetry_endpoint .. "/visit", { - timeout = 1000, - body = vim.json.encode(json), - headers = { - content_type = "application/json", - }, - }) - if res.status ~= 200 then - print("Error sending data to the restapi via the endpoint " .. telemetry_endpoint .. "/visit") - end - utils.verbose_print("Data sent to the restapi via the telemetry endpoint for file " .. filepath) - end + local res = curl.post(M.config.telemetry_endpoint .. "/visit", { + timeout = 1000, + body = vim.json.encode(json), + headers = { + content_type = "application/json", + }, + }) + if res.status ~= 200 then + print("Error sending data to the restapi via the endpoint " .. M.config.telemetry_endpoint .. "/visit") + end + utils.verbose_print("Data sent to the restapi via the telemetry endpoint for file " .. filepath) end -local function remove_data_from_telemetry_db(filepath, entry_timestamp, exit_timestamp) - -end +---TODO: This is not implemented yet +---@diagnostic disable-next-line: unused-function, unused-local +local function remove_data_from_telemetry_db(filepath, entry_timestamp, exit_timestamp) end ---- Start the timer for the current buffer --- Happens when we enter to a buffer +--- Start the timer for the current buffer, called when entering the buffer +---@param bufnr number function M.start_timer(bufnr) local filepath = vim.api.nvim_buf_get_name(bufnr) @@ -93,37 +123,46 @@ function M.start_timer(bufnr) current_bufnr = bufnr current_buffer_filepath = filepath - -- Do not log an "empty" buffer - -- if filepath == "" then - -- utils.verbose_print("Filename is '' so we are not logging this buffer") - -- return - -- end - local git_project_name = utils.get_git_project_name() + local git_branch = utils.get_git_branch() local buffer_filetype = utils.get_buffer_filetype(bufnr) if not usage_data.data[filepath] then + ---@type UsagedataPerFilepath usage_data.data[filepath] = { git_project_name = git_project_name, + git_branch = git_branch, filetype = buffer_filetype, - -- Will be populated with entries like this: { entry = os.time(), exit = nil , elapsed_time_sec = 0, keystrokes = 0 } - visit_log = {} + -- Will be populated with entries like this: + -- { entry = os.time(), + -- exit = nil , + -- elapsed_time_sec = 0, + -- keystrokes = 0 } + --- @type VisitLogEntry[] + visit_log = {}, } end -- TODO: should we notify the user if the git project name has changed? usage_data.data[filepath].git_project_name = git_project_name + usage_data.data[filepath].git_branch = git_branch -- Record an entry event - usage_data.data[filepath].visit_log[#usage_data.data[filepath].visit_log + 1] = { + table.insert(usage_data.data[filepath].visit_log, { entry = os.time(), exit = nil, keystrokes = 0, - elapsed_time_sec = 0 - } - - utils.verbose_print("Timer started for " .. - current_buffer_filepath .. " (buffer " .. current_bufnr .. ") at " .. os.date("%c", os.time())) + elapsed_time_sec = 0, + }) + + utils.verbose_print( + "Timer started for " + .. current_buffer_filepath + .. " (buffer " + .. current_bufnr + .. ") at " + .. os.date("%c", os.time()) + ) -- Save the updated time to the JSON file save_usage_data() @@ -135,8 +174,6 @@ end ---Used when inactivity was detected, as we don't want to log the inactive time ---@return nil function M.stop_timer(use_last_activity) - use_last_activity = use_last_activity or false - local filepath = current_buffer_filepath local current_time @@ -151,35 +188,49 @@ function M.stop_timer(use_last_activity) -- and calculate the elapsed time -- Save entry and exit event only if the elapsed time between them is more than N seconds local visit_log = usage_data.data[filepath].visit_log - if (#visit_log > 0) and ((current_time - visit_log[#visit_log].entry) > vim.g.usagetracker_event_wait_period_in_sec) then + if (#visit_log > 0) and ((current_time - visit_log[#visit_log].entry) > M.config.event_wait_period_in_sec) then local last_entry = visit_log[#visit_log] last_entry.exit = current_time last_entry.elapsed_time_sec = last_entry.exit - last_entry.entry -- Send data to the restapi - send_data_to_restapi(filepath, + send_data_to_restapi( + filepath, last_entry.entry, last_entry.exit, last_entry.keystrokes, usage_data.data[filepath].filetype, - usage_data.data[filepath].git_project_name) + usage_data.data[filepath].git_project_name, + usage_data.data[filepath].git_branch + ) else - utils.verbose_print("Not saving the last entry event for " .. - filepath .. - " as the elapsed time is less than " .. vim.g.usagetracker_event_wait_period_in_sec .. " seconds") + utils.verbose_print( + "Not saving the last entry event for " + .. filepath + .. " as the elapsed time is less than " + .. M.config.event_wait_period_in_sec + .. " seconds" + ) -- Remove the last entry event visit_log[#visit_log] = nil end end - utils.verbose_print("Timer stopped for " .. - current_buffer_filepath .. " (buffer " .. current_bufnr .. ") at " .. os.date("%c", current_time)) + utils.verbose_print( + "Timer stopped for " + .. current_buffer_filepath + .. " (buffer " + .. current_bufnr + .. ") at " + .. os.date("%c", current_time) + ) -- Save the updated time to the JSON file save_usage_data() end --- Count the keystrokes +--- Count the keystrokes +---@param bufnr number function M.activity_on_keystroke(bufnr) local filepath = vim.api.nvim_buf_get_name(bufnr) @@ -188,10 +239,6 @@ function M.activity_on_keystroke(bufnr) M.start_timer(bufnr) end - -- if filepath == "" then - -- return - -- end - if usage_data.data[filepath] then local visit_log = usage_data.data[filepath].visit_log if #visit_log > 0 then @@ -218,7 +265,6 @@ function M.show_lifetime_usage_by_file() local headers = { "Filepath", "Keystrokes", "Time (min)", "Project", "Filetype" } local field_names = { "path", "keystrokes", "elapsed_time_in_min", "git_project_name", "filetype" } - -- Sort the result table based on elapsed_time_sec in descending order table.sort(result, function(a, b) return a.elapsed_time_in_min > b.elapsed_time_in_min @@ -228,6 +274,7 @@ function M.show_lifetime_usage_by_file() draw.print_table_format(headers, result, field_names) end +---@param filepath string|nil function M.show_visit_log(filepath) if filepath == nil then filepath = vim.api.nvim_buf_get_name(vim.api.nvim_get_current_buf()) @@ -239,34 +286,48 @@ function M.show_visit_log(filepath) local headers = { "Filepath", "Enter", "Exit", "Time (min)", "Keystrokes" } local field_names = { "filepath", "enter", "exit", "elapsed_time_in_min", "keystrokes" } + --- convert timestamp to data + ---@param ts number + ---@return string local function ts_to_date(ts) - return os.date("%Y-%m-%d %H:%M:%S", ts) + return tostring(os.date("%Y-%m-%d %H:%M:%S", ts)) + end + + --- helper function to be used in the two cases below + ---@param filep string + local function visit_log_helper(visit, filep) + ---@type string + local enter = ts_to_date(visit.entry) + ---@type string + local exit + ---@type number + local elapsed_time_in_min + if visit.exit == nil then + exit = "Present" + elapsed_time_in_min = math.floor((os.time() - visit.entry) / 60 * 100) / 100 + else + exit = ts_to_date(visit.exit) + elapsed_time_in_min = math.floor((visit.exit - visit.entry) / 60 * 100) / 100 + end + table.insert(visit_log_table, { + filepath = filep, + enter = enter, + exit = exit, + elapsed_time_in_min = elapsed_time_in_min, + keystrokes = visit.keystrokes, + }) end if not usage_data.data[filepath] then - print("No visit log for this file (filepath: " .. - filepath .. "). Instead showing all the visit logs from all the files.") + print( + "No visit log for this file (filepath: " + .. filepath + .. "). Instead showing all the visit logs from all the files." + ) -- Instead show all the visit logs from all the files - for f, file_visit_logs in pairs(usage_data.data) do for _, visit in ipairs(file_visit_logs.visit_log) do - local enter = ts_to_date(visit.entry) - local exit - local elapsed_time_in_min - if visit.exit == nil then - exit = "Present" - elapsed_time_in_min = math.floor((os.time() - visit.entry) / 60 * 100) / 100 - else - exit = ts_to_date(visit.exit) - elapsed_time_in_min = math.floor((visit.exit - visit.entry) / 60 * 100) / 100 - end - visit_log_table[#visit_log_table + 1] = { - filepath = f, - enter = enter, - exit = exit, - elapsed_time_in_min = elapsed_time_in_min, - keystrokes = visit.keystrokes - } + visit_log_helper(visit, f) end end else @@ -275,23 +336,7 @@ function M.show_visit_log(filepath) -- Convert the visit log to a table for i, row in ipairs(visit_log) do if i <= #visit_log then - local enter = ts_to_date(row.entry) - local exit = nil - local elapsed_time_in_min = nil - if row.exit == nil then - exit = "Present" - elapsed_time_in_min = math.floor((os.time() - row.entry) / 60 * 100) / 100 - else - exit = ts_to_date(row.exit) - elapsed_time_in_min = math.floor((row.exit - row.entry) / 60 * 100) / 100 - end - visit_log_table[#visit_log_table + 1] = { - filepath = filepath, - enter = enter, - exit = exit, - elapsed_time_in_min = elapsed_time_in_min, - keystrokes = row.keystrokes - } + visit_log_helper(row, filepath) end end end @@ -305,6 +350,8 @@ function M.show_visit_log(filepath) draw.print_table_format(headers, visit_log_table, field_names) end +---@param filetypes string[]? Filetypes which we would like to include, if empty then we don't filter for any filetypes and everything is included +---@param project_name string? Project name which we would like to include, if empty then we don't filter for any project and everything is included function M.show_daily_stats(filetypes, project_name) local data = agg.create_daily_usage_aggregation(usage_data, filetypes, project_name) if #data == 0 then @@ -314,10 +361,10 @@ function M.show_daily_stats(filetypes, project_name) local barchart_data = {} for _, item in ipairs(data) do - barchart_data[#barchart_data + 1] = { + table.insert(barchart_data, { name = item.day_str, - value = item.time_in_min - } + value = item.time_in_min, + }) end local title = "Daily usage in minutes" @@ -330,49 +377,64 @@ function M.show_daily_stats(filetypes, project_name) draw.vertical_barchart(barchart_data, 60, title, false, 42) end -function M.show_aggregation(key, start_date_str, end_date_str) - local valid_keys = { - filetype = true, - project = true, - filepath = true - } - - if not key then - print("Please specify an aggregation key: filetype, project, or filepath") - return - end - - if not valid_keys[key] then - print("Invalid aggregation key: " .. key .. ". Valid keys are: filetype, project, filepath") - return - end - - local today = os.date("%Y-%m-%d") - local start_date_str = start_date_str or today - local start_date_timestamp = utils.convert_string_to_date(start_date_str) +--- Aggregation in the message +---@param key string +---@param _start_date? string +---@param _end_date? string +function M.show_aggregation(key, _start_date, _end_date) + local function show_agg_helper(_key) + local today = tostring(os.date("%Y-%m-%d")) + local start_date_str = _start_date or today + local start_date_timestamp = utils.convert_string_to_date(start_date_str) + + local tomorrow = tostring(os.date("%Y-%m-%d", utils.increment_timestamp_by_days(os.time(), 1))) + local end_date_str = _end_date or tomorrow + local end_date_timestamp = utils.convert_string_to_date(end_date_str) + if not start_date_timestamp or not end_date_timestamp then + return + end + local data = agg.aggregate(usage_data, _key, start_date_timestamp, end_date_timestamp) - local tomorrow = os.date("%Y-%m-%d", utils.increment_timestamp_by_days(os.time(), 1)) - local end_date_str = end_date_str or tomorrow - local end_date_timestamp = utils.convert_string_to_date(end_date_str) + if not data or next(data) == nil then + print("No data to show - try using different filters") + return + end - local data = agg.aggregate(usage_data, key, start_date_timestamp, end_date_timestamp) + local barchart_data = {} + for _, item in ipairs(data) do + local value = math.floor(item.time_in_sec / 60 * 100) / 100 + table.insert(barchart_data, { + name = item.name, + value = value, + }) + end - if not data or next(data) == nil then - print("No data to show - try using different filters") - return + local title = "Total usage in minutes from " .. start_date_str .. " 00:00 to " .. end_date_str .. " 00:00" + draw.vertical_barchart(barchart_data, 60, title, true, 42) end - local barchart_data = {} - for _, item in ipairs(data) do - local value = math.floor(item.time_in_sec / 60 * 100) / 100 - barchart_data[#barchart_data + 1] = { - name = item.name, - value = value - } + local valid_keys = { "filetype", "project", "filepath" } + if key and not vim.tbl_contains(valid_keys, key) then + vim.notify("UsageTracker: Aggregation key " .. key .. " is not valid", vim.log.levels.WARN) + key = "" end - local title = "Total usage in minutes from " .. start_date_str .. " 00:00 to " .. end_date_str .. " 00:00" - draw.vertical_barchart(barchart_data, 60, title, true, 42) + if not key or key == "" then + vim.ui.select({ "filetype", "project", "filepath" }, { + prompt = "Choose an aggregation key:", + format_item = function(item) + return string.format("Aggregate by %s", item) + end, + }, function(choice) + if choice then + show_agg_helper(choice) + else + vim.notify("UsageTracker: No aggregation key was chosen. Do nothing.", vim.log.levels.WARN) + end + end) + else + show_agg_helper(key) + end end --- Remove and item from the visit log based on the filepath, entry timestamp, and exit timestamp @@ -381,7 +443,7 @@ end ---@param exit_timestamp number function M.remove_entry_from_visit_log(filepath, entry_timestamp, exit_timestamp) -- This function can only run with an empty buffer where the filename is empty ('') - if current_buffer_filepath ~= '' then + if current_buffer_filepath ~= "" then print("Please run this function with an empty buffer") return end @@ -391,8 +453,12 @@ function M.remove_entry_from_visit_log(filepath, entry_timestamp, exit_timestamp return end - entry_timestamp = tonumber(entry_timestamp) - exit_timestamp = tonumber(exit_timestamp) + if type(entry_timestamp) ~= "number" then + return + end + if type(exit_timestamp) ~= "number" then + return + end local removed_item = false @@ -405,8 +471,14 @@ function M.remove_entry_from_visit_log(filepath, entry_timestamp, exit_timestamp -- Update the visit_log in the usage_data table usage_data.data[filepath].visit_log = visit_log removed_item = true - print("Removed entry from visit log for " .. - filepath .. " with entry timestamp " .. entry_timestamp .. " and exit timestamp " .. exit_timestamp) + print( + "Removed entry from visit log for " + .. filepath + .. " with entry timestamp " + .. entry_timestamp + .. " and exit timestamp " + .. exit_timestamp + ) break end end @@ -418,15 +490,21 @@ function M.remove_entry_from_visit_log(filepath, entry_timestamp, exit_timestamp if removed_item then save_usage_data() else - print("No entry found for filepath: " .. filepath .. " with entry timestamp " .. entry_timestamp .. - " and exit timestamp " .. exit_timestamp) + print( + "No entry found for filepath: " + .. filepath + .. " with entry timestamp " + .. entry_timestamp + .. " and exit timestamp " + .. exit_timestamp + ) end end --- Sometimes the visit log can have bad entries where the logged time is just too much -- This function removes them from the log based on a usage threshold ---@param logged_minute_threshold number The threshold in minutes for the logged time -function M.clenup_log_from_bad_entries(logged_minute_threshold) +function M.cleanup_log_from_bad_entries(logged_minute_threshold) if not logged_minute_threshold then print("Please provide a logged minute threshold") return @@ -444,15 +522,27 @@ function M.clenup_log_from_bad_entries(logged_minute_threshold) -- if exit timestamp is not set then let's set it for now (this can happen when a timer is not stopped) if not row.exit then row.exit = os.time() - print("Exit timestamp was not set for " .. filepath .. " with entry timestamp " .. row.entry .. - " - setting it to " .. row.exit) + print( + "Exit timestamp was not set for " + .. filepath + .. " with entry timestamp " + .. row.entry + .. " - setting it to " + .. row.exit + ) end local elapsed_time_in_sec = row.exit - row.entry if elapsed_time_in_sec > time_threshold_in_sec then table.remove(visit_log, i) removed_items = removed_items + 1 - print("Removed entry from visit log for " .. filepath .. " with entry timestamp " .. row.entry .. - " and exit timestamp " .. row.exit) + print( + "Removed entry from visit log for " + .. filepath + .. " with entry timestamp " + .. row.entry + .. " and exit timestamp " + .. row.exit + ) else i = i + 1 end @@ -461,17 +551,23 @@ function M.clenup_log_from_bad_entries(logged_minute_threshold) end print("Removed " .. removed_items .. " items from the local visit log") - local telemetry_endpoint = vim.g.usagetracker_telemetry_endpoint + local telemetry_endpoint = M.config.telemetry_endpoint if telemetry_endpoint and telemetry_endpoint ~= "" then print("Removing entries from the Telemetry DB...") local url = telemetry_endpoint .. "/cleanup?threshold_in_min=" .. logged_minute_threshold local response = curl.delete(url, { accept = "application/json", timeout = 1000 }) if response.status == 200 then local data = vim.json.decode(response.body) - if data.entries then + if data and data.entries then for _, entry in ipairs(data.entries) do - print("Removed entry from telemetry DB for " .. entry.filepath .. " with entry timestamp " .. - entry.entry .. " and exit timestamp " .. entry.exit) + print( + "Removed entry from telemetry DB for " + .. entry.filepath + .. " with entry timestamp " + .. entry.entry + .. " and exit timestamp " + .. entry.exit + ) end print("Removed " .. #data.entries .. " items from the telemetry DB") else @@ -486,7 +582,7 @@ function M.clenup_log_from_bad_entries(logged_minute_threshold) end -- Clean up the visit log by removing older than 2 week entries (where the entry is older than 2 weeks) -local function clenup_visit_log(filepath, days) +local function cleanup_visit_log(filepath, days) local visit_log = usage_data.data[filepath].visit_log local now = os.time() local time_threshold_in_sec = days * 24 * 60 * 60 @@ -507,37 +603,24 @@ local function handle_inactivity() return end - if (os.time() - last_activity_timestamp) > (vim.g.usagetracker_inactivity_threshold_in_min * 60) then + if (os.time() - last_activity_timestamp) > (M.config.inactivity_threshold_in_min * 60) then -- Stop the timer for the current buffer utils.verbose_print("Stopping due to inactivity") M.stop_timer(true) is_inactive = true - print("Inactivity detected for buffer " .. - current_bufnr .. " at " .. os.date("%Y-%m-%d %H:%M:%S") .. ", last active at " .. last_activity_timestamp) + print( + "Inactivity detected for buffer " + .. current_bufnr + .. " at " + .. os.date("%Y-%m-%d %H:%M:%S") + .. ", last active at " + .. last_activity_timestamp + ) end end function M.setup(opts) - -- Plugin parameters -- - - local function set_default(opt, default) - local prefix = "usagetracker_" - if vim.g[prefix .. opt] ~= nil then - return - elseif opts[opt] ~= nil then - vim.g[prefix .. opt] = opts[opt] - else - vim.g[prefix .. opt] = default - end - end - - set_default("keep_eventlog_days", 14) - set_default("cleanup_freq_days", 7) - set_default("event_wait_period_in_sec", 5) - set_default("inactivity_threshold_in_min", 2) - set_default("inactivity_check_freq_in_sec", 1) - set_default("verbose", 0) - set_default("telemetry_endpoint", "") + require("usage-tracker.config").setup_config(opts) -- Initialize some of the "global" variables last_activity_timestamp = os.time() @@ -545,85 +628,118 @@ function M.setup(opts) current_buffer_filepath = vim.api.nvim_buf_get_name(current_bufnr) -- Load existing data -- - load_usage_data() -- Load the timers from the JSON file on plugin setup - -- Autocmd -- - - vim.api.nvim_exec([[ - augroup UsageTracker - autocmd! - - autocmd BufEnter * lua require('usage-tracker').start_timer(vim.api.nvim_get_current_buf()) - autocmd BufLeave,QuitPre * lua require('usage-tracker').stop_timer(false) - - autocmd TextChanged,TextChangedI * lua require('usage-tracker').activity_on_keystroke(vim.api.nvim_get_current_buf()) - autocmd CursorMoved,CursorMovedI * lua require('usage-tracker').activity_on_keystroke(vim.api.nvim_get_current_buf()) - augroup END - ]], false) - - - -- Commands -- + -- Telemetry check -- + -- Tell the user that the service is not running if there is a set endpoint + if M.config.telemetry_endpoint and M.config.telemetry_endpoint ~= "" then + local res = curl.get(M.config.telemetry_endpoint .. "/status", { + timeout = 500, + on_error = function() end, + }) - vim.api.nvim_create_user_command("UsageTrackerShowFilesLifetime", - function() - M.show_lifetime_usage_by_file() - end, - {}) + if res.status ~= 200 then + vim.api.nvim_echo({ + { + "UsageTracker: ", + "WarningMsg", + }, + { + "Telemetry service is configured, but server is not responding: " + .. M.config.telemetry_endpoint + .. "/status" + .. ".. Turning off telemetry service...", + "", + }, + }, true, {}) + M.config.telemetry_endpoint = "" + end + end - vim.api.nvim_create_user_command("UsageTrackerShowVisitLog", - function(cmd_opts) - M.show_visit_log(cmd_opts.fargs[1] or nil) + -- Autocmd -- + local augroup_id = vim.api.nvim_create_augroup("UsageTracker", { clear = true }) + + -- TODO: How about these events? + -- autocmd BufEnter,VimEnter + -- autocmd CursorHold,CursorHoldI + -- autocmd BufWritePost + -- autocmd QuitPre * call s:SendHeartbeats() + + vim.api.nvim_create_autocmd("BufEnter", { + group = augroup_id, + pattern = "*", + callback = function() + require("usage-tracker").start_timer(vim.api.nvim_get_current_buf()) end, - { nargs = '?' }) + }) - vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregation", - function() - M.show_daily_stats(nil, nil) + vim.api.nvim_create_autocmd({ "BufLeave", "QuitPre" }, { + group = augroup_id, + pattern = "*", + callback = function() + require("usage-tracker").stop_timer(false) end, - {}) + }) - vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregationByFiletypes", - function(cmd_opts) - local filetypes = cmd_opts.fargs - if #filetypes == 0 then - filetypes = nil - end - M.show_daily_stats(filetypes, nil) + vim.api.nvim_create_autocmd({ "TextChanged", "TextChangedI", "CursorMoved", "CursorMovedI" }, { + group = augroup_id, + pattern = "*", + callback = function() + require("usage-tracker").activity_on_keystroke(vim.api.nvim_get_current_buf()) end, - { nargs = '*' }) + }) - vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregationByProject", - function(cmd_opts) - M.show_daily_stats(nil, cmd_opts.fargs[1] or nil) - end, - { nargs = '?' }) - - vim.api.nvim_create_user_command("UsageTrackerShowAgg", - function(cmd_opts) - M.show_aggregation(cmd_opts.fargs[1], cmd_opts.fargs[2] or nil, cmd_opts.fargs[3] or nil) - end, - { nargs = '*' }) - vim.api.nvim_create_user_command("UsageTrackerRemoveEntry", - function(cmd_opts) - M.remove_entry_from_visit_log(cmd_opts.fargs[1], cmd_opts.fargs[2], cmd_opts.fargs[3]) - end, - { nargs = '*' }) - vim.api.nvim_create_user_command("UsageTrackerClenup", - function(cmd_opts) - M.clenup_log_from_bad_entries(cmd_opts.fargs[1] or nil) + -- Commands -- + vim.api.nvim_create_user_command("UsageTrackerShowFilesLifetime", function() + M.show_lifetime_usage_by_file() + end, {}) + + vim.api.nvim_create_user_command("UsageTrackerShowVisitLog", function(cmd_opts) + M.show_visit_log(cmd_opts.fargs[1] or nil) + end, { nargs = "?" }) + + vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregation", function() + M.show_daily_stats(nil, nil) + end, {}) + + vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregationByFiletypes", function(cmd_opts) + local filetypes = cmd_opts.fargs + if #filetypes == 0 then + filetypes = nil + end + M.show_daily_stats(filetypes, nil) + end, { nargs = "*" }) + + vim.api.nvim_create_user_command("UsageTrackerShowDailyAggregationByProject", function(cmd_opts) + M.show_daily_stats(nil, cmd_opts.fargs[1] or nil) + end, { nargs = "?" }) + + vim.api.nvim_create_user_command("UsageTrackerShowAgg", function(cmd_opts) + M.show_aggregation(cmd_opts.fargs[1], cmd_opts.fargs[2] or nil, cmd_opts.fargs[3] or nil) + end, { + nargs = "*", + complete = function(_, _) + return { "filetype", "project", "filepath" } end, - { nargs = '?' }) - + }) + vim.api.nvim_create_user_command("UsageTrackerRemoveEntry", function(cmd_opts) + M.remove_entry_from_visit_log(cmd_opts.fargs[1], cmd_opts.fargs[2], cmd_opts.fargs[3]) + end, { nargs = "*" }) + vim.api.nvim_create_user_command("UsageTrackerCleanup", function(cmd_opts) + M.cleanup_log_from_bad_entries(cmd_opts.fargs[1]) + end, { nargs = "?" }) - -- Cleanup -- + vim.api.nvim_create_user_command("UsageTrackerHTML", function() + require("usage-tracker.html").create_html_stats() + end, { nargs = "?" }) -- Clean up the visit log + --TODO: Upon cleanup, the old logs should be preserverd with a suffix local now = os.time() - if now - usage_data.last_cleanup > (vim.g.usagetracker_cleanup_freq_days * 24 * 60 * 60) then + if now - usage_data.last_cleanup > (M.config.cleanup_freq_days * 24 * 60 * 60) then for filepath, _ in pairs(usage_data.data) do - clenup_visit_log(filepath, vim.g.usagetracker_keep_eventlog_days) + cleanup_visit_log(filepath, M.config.keep_eventlog_days) end usage_data.last_cleanup = now save_usage_data() @@ -631,27 +747,13 @@ function M.setup(opts) -- Check for inactivity every N seconds local timer = vim.loop.new_timer() - timer:start(0, - vim.g.usagetracker_inactivity_check_freq_in_sec * 1000, + timer:start( + 0, + M.config.inactivity_check_freq_in_sec * 1000, vim.schedule_wrap(function() handle_inactivity() end) ) - - -- Telemetry check -- - -- Tell the user that the service is not running if there is a set endpoint - if vim.g.usagetracker_telemetry_endpoint and vim.g.usagetracker_telemetry_endpoint ~= "" then - local success, res = pcall(function() - return curl.get(vim.g.usagetracker_telemetry_endpoint .. "/status", { - timeout = 1000 }) - end) - if not success or res.status ~= 200 then - print("UsageTracker: Telemetry service is enabled but not running: " .. - vim.g.usagetracker_telemetry_endpoint .. "/status") - print("UsageTracker: Turning off telemetry service...") - vim.g.usagetracker_telemetry_endpoint = nil - end - end end return M diff --git a/lua/usage-tracker/utils.lua b/lua/usage-tracker/utils.lua index 2e26538..25e6928 100644 --- a/lua/usage-tracker/utils.lua +++ b/lua/usage-tracker/utils.lua @@ -2,7 +2,8 @@ local M = {} ---@param message string function M.verbose_print(message) - if vim.g.usagetracker_verbose > 0 then + local verbose = require("usage-tracker.config").config.verbose + if verbose and verbose > 0 then print("[usage-tracker.nvim]: " .. message) end end @@ -30,7 +31,7 @@ end --- Return a date object from a timestamp ---@param timestamp number ---@param keep_day_only boolean If true, the hour, minute and second will be set to 0 ----@return osdate +---@return osdate|string function M.timestamp_to_date(timestamp, keep_day_only) local d = os.date("*t", timestamp) if keep_day_only then @@ -45,6 +46,7 @@ end ---@param date osdate ---@return number Timestamp function M.date_to_timestamp(date) + ---@diagnostic disable-next-line: param-type-mismatch return os.time(date) end @@ -55,26 +57,38 @@ end function M.increment_timestamp_by_days(timestamp, days) local increased_date = os.date("*t", timestamp) increased_date.day = increased_date.day + days + ---@diagnostic disable-next-line: param-type-mismatch local increased_timestamp = os.time(increased_date) return increased_timestamp end ---- Get the current git project name --- If the file is not in a git project, return an empty string +---@return string function M.get_git_project_name() - local result = vim.fn.systemlist('git rev-parse --show-toplevel 2>/dev/null') - if vim.v.shell_error == 0 and result[1] ~= '' then + local result = vim.fn.systemlist("git rev-parse --show-toplevel 2>/dev/null") + -- If the file is not in a git project, return an empty string + if vim.v.shell_error == 0 and result ~= nil and result[1] ~= "" then local folder_path = vim.trim(result[1]) - return vim.fn.fnamemodify(folder_path, ":t") + return tostring(vim.fn.fnamemodify(folder_path, ":t")) else - return '' + return "" + end +end + +---@return string +function M.get_git_branch() + local result = vim.fn.systemlist("git branch --show-current 2>/dev/null") + if vim.v.shell_error == 0 and result ~= nil and result[1] ~= "" then + local folder_path = vim.trim(result[1]) + return tostring(vim.fn.fnamemodify(folder_path, ":t")) + else + return "" end end ---- Get the git project name ---@param bufnr integer +---@return string function M.get_buffer_filetype(bufnr) - local filetype = vim.api.nvim_buf_get_option(bufnr, "filetype") + local filetype = vim.api.nvim_get_option_value("ft", { buf = bufnr }) if filetype == "" then return "" else @@ -84,21 +98,21 @@ end --- Parse a date string like this 2022-06-12 to a timestamp ---@param str string The date string ----@return number The timestamp +---@return number|nil The timestamp function M.convert_string_to_date(str) local year, month, day = str:match("(%d+)-(%d+)-(%d+)") + -- Convert the string components to numbers + year = tonumber(year) + month = tonumber(month) + day = tonumber(day) + -- Check if the date components are valid if not year or not month or not day then print("Invalid date format. This is the acepted format: YYYY-MM-DD, like 2022-06-12") return nil end - -- Convert the string components to numbers - year = tonumber(year) - month = tonumber(month) - day = tonumber(day) - local parsed_date_timestamp = os.time({ year = year, month = month, day = day, hour = 0, min = 0, sec = 0 }) -- Check if the date is valid using os.time diff --git a/screenshot.webp b/screenshot.webp new file mode 100644 index 0000000..f58987c Binary files /dev/null and b/screenshot.webp differ diff --git a/telemetry_api/Dockerfile b/telemetry_api/Dockerfile index ee3abcc..363c11b 100644 --- a/telemetry_api/Dockerfile +++ b/telemetry_api/Dockerfile @@ -1,17 +1,13 @@ -FROM python:3.9-slim-buster - +FROM python:3-slim WORKDIR /app - COPY requirements.txt . - -RUN pip install --upgrade pip -RUN pip install --no-cache-dir -r requirements.txt - +ENV VIRTUAL_ENV=/opt/venv +RUN python -m venv $VIRTUAL_ENV +ENV PATH="$VIRTUAL_ENV/bin:$PATH" +RUN --mount=type=cache,target=/root/.cache pip install --upgrade pip \ + && pip install -r ./requirements.txt COPY . . - VOLUME /app/data - EXPOSE 8000 - CMD ["uvicorn", "restapi:app", "--host", "0.0.0.0", "--port", "8000"] diff --git a/telemetry_api/README.md b/telemetry_api/README.md index 643308e..7f26a1d 100644 --- a/telemetry_api/README.md +++ b/telemetry_api/README.md @@ -1,7 +1,7 @@ # Telemetry endpoint This small restapi is responsible for the data collection if you enabled telemetry by defining the endpoint -in the `setup({..., telemetry_endpoint="http://localhost:8000/visit"})` +in the `setup({..., telemetry_endpoint="http://localhost:8000"})` # Setup @@ -9,12 +9,12 @@ Change the volume mount if it is required ## Build & Run -``` +```bash # Build -docker-compose build +docker compose build # Run - it should always start (until you manually stop it) -docker-compose up -d +docker compose up -d ``` # Queries diff --git a/telemetry_api/docker-compose.yml b/telemetry_api/docker-compose.yml index ae65a06..550305c 100644 --- a/telemetry_api/docker-compose.yml +++ b/telemetry_api/docker-compose.yml @@ -6,7 +6,7 @@ services: context: . dockerfile: Dockerfile ports: - - 8999:8000 + - 8000:8000 volumes: - ./sqlite_db:/app/data restart: always diff --git a/telemetry_api/restapi.py b/telemetry_api/restapi.py index b1d9b59..fabdb8b 100644 --- a/telemetry_api/restapi.py +++ b/telemetry_api/restapi.py @@ -29,7 +29,8 @@ id INTEGER PRIMARY KEY AUTOINCREMENT, filepath TEXT UNIQUE NOT NULL, filetype TEXT, - projectname TEXT, + git_project_name TEXT, + git_branch TEXT, lastmodification INTEGER NOT NULL ) """) @@ -43,13 +44,15 @@ class Visit(BaseModel): keystrokes: int filepath: str filetype: Optional[str] - projectname: Optional[str] + git_project_name: Optional[str] + git_branch: Optional[str] class FileInfo(BaseModel): filepath: str filetype: Optional[str] - projectname: Optional[str] + git_project_name: Optional[str] + git_branch: Optional[str] lastmodification: int @@ -69,10 +72,11 @@ async def create_visit(visit: Visit): else: file_info = FileInfo(filepath=visit.filepath, filetype=visit.filetype, - projectname=visit.projectname, + git_project_name=visit.git_project_name, + git_branch=visit.git_branch, lastmodification=visit.exit) - c.execute("INSERT INTO fileinfo (filepath, filetype, projectname, lastmodification) VALUES (?, ?, ?, ?)", - (file_info.filepath, file_info.filetype, file_info.projectname, file_info.lastmodification)) + c.execute("INSERT INTO fileinfo (filepath, filetype, git_project_name, git_branch, lastmodification) VALUES (?, ?, ?, ?, ?)", + (file_info.filepath, file_info.filetype, file_info.git_project_name, file_info.git_branch, file_info.lastmodification)) c.execute("INSERT INTO visits (entry, exit, keystrokes, filepath) VALUES (?, ?, ?, ?)", (visit.entry, visit.exit, visit.keystrokes, visit.filepath)) diff --git a/tests/config_spec.lua b/tests/config_spec.lua new file mode 100644 index 0000000..51eda57 --- /dev/null +++ b/tests/config_spec.lua @@ -0,0 +1,46 @@ +describe("Config module", function() + local usage + + it("The module can be required", function() + local your_plugin_path = "/home/konrad/nvimplugins/usage-tracker.nvim" + package.path = package.path .. ";" .. your_plugin_path .. "/lua/?.lua" + package.path = package.path .. ";" .. your_plugin_path .. "/lua/?/init.lua" + -- print("########################:" .. (package.searchpath("usage-tracker", package.path) or "nothing found")) + usage = require("usage-tracker") + -- local status, plugin = pcall(require, "usage-tracker") + -- assert.True(status, "The plugin should be successfully required.") + assert.truthy(usage, "The plugin should be successfully required.") + end) + + before_each(function() + package.loaded["usage-tracker"] = nil + package.loaded["usage-tracker.config"] = nil + usage = nil + usage = require("usage-tracker") + end) + + it("config table is the same as in the config submodule", function() + assert.equal(usage.config, require("usage-tracker.config").config) + end) + + it("setup config works when only one element given", function() + usage.setup({ verbose = 1 }) + assert.equal(1, usage.config.verbose, "given parameter is set") + assert.equal(7, usage.config.cleanup_freq_days, "other parameters are on default values") + end) + + it("Default config", function() + assert.truthy(usage, "requirements are kept between tests") + local test_config = { + keep_eventlog_days = 4, + cleanup_freq_days = 7, + event_wait_period_in_sec = 5, + inactivity_threshold_in_min = 2, + inactivity_check_freq_in_sec = 1, + verbose = 0, + telemetry_endpoint = "", + json_file = vim.fn.stdpath("config") .. "/usage_data.json", + } + assert.are.same(test_config, usage.config, "configuration is as expected") + end) +end)