Skip to content

E2E Tests version 1.0 - #2

Merged
izzywdev merged 159 commits into
e2e-deployfrom
master
Jan 7, 2025
Merged

izzywdev merged 159 commits into
e2e-deployfrom
master

Conversation

@izzywdev

@izzywdev izzywdev commented Jan 7, 2025

Copy link
Copy Markdown
Owner

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

  • I sign off on contributing this submission to open-source
  • My code follows the code style of this project.
  • My change requires changes to the documentation.
  • I have updated the documentation accordingly.
  • All new and existing tests passed.
  • This PR does not contain plagiarized content.
  • The title of my pull request is a short description of the requested changes.

Screenshots

Note to reviewers

tobihans and others added 30 commits October 15, 2024 17:13
This commit introduces remote debugging capabilities and enhances
the authorization process in the sample service. Key changes include:

- Add debugpy for remote debugging support
- Expose port 5682 for debug connections
- Refine OPAL authorization response handling
- Update nginx configuration for better request processing
- Modify Flask app startup to support debugging
- Update dependencies in requirements.txt

These changes improve development workflow and strengthen the
authentication mechanism, allowing for more robust testing and
troubleshooting of the authorization process.
This change introduces a Gitea service to the local development
environment, providing a self-hosted Git repository solution.
The OPAL server is now configured to fetch policies from this
local Gitea instance instead of GitHub, enhancing control and
security for policy management in development scenarios.

The commit also includes minor code cleanup in the sample
service app, removing an unnecessary blank line.

These modifications aim to improve the development workflow
and provide better isolation for testing policy changes.
The URL for the OPAL policy repository has been modified to reflect
a change in the organization name from 'permitio' to 'permit'. This
ensures that the system points to the correct repository location,
maintaining proper functionality of policy fetching and updates.

Issue: #
	new file:   CONTRIBUTING.md
	new file:   LICENSE
	new file:   MANIFEST.in
	new file:   Makefile
	new file:   README.md
	new file:   app-tests/README.md
	new file:   app-tests/docker-compose-app-tests.yml
	new file:   app-tests/run.sh
	new file:   docker/Dockerfile
	new file:   docker/docker-compose-api-policy-source-example.yml
	new file:   docker/docker-compose-example-cedar.yml
	new file:   docker/docker-compose-example.yml
	new file:   docker/docker-compose-git-webhook.yml
	new file:   docker/docker-compose-scopes-example.yml
	new file:   docker/docker-compose-with-callbacks.yml
	new file:   docker/docker-compose-with-kafka-example.yml
	new file:   docker/docker-compose-with-oauth-initial.yml
	new file:   docker/docker-compose-with-rate-limiting.yml
	new file:   docker/docker-compose-with-security.yml
	new file:   docker/docker-compose-with-statistics.yml
	new file:   docker/docker_files/bundle_files/bundle.tar.gz
	new file:   docker/docker_files/bundle_files/bundle.tar.gz.bak
	new file:   docker/docker_files/cedar_data/data.json
	new file:   docker/docker_files/nginx.conf
	new file:   docker/docker_files/policy_test/authz.rego
	new file:   docker/run-example-with-scopes.sh
	new file:   docker/run-example-with-security.sh
	new file:   documentation/.gitignore
	new file:   documentation/babel.config.js
	new file:   documentation/docs/FAQ.mdx
	new file:   documentation/docs/fetch-providers.mdx
	new file:   documentation/docs/getting-started/configuration.mdx
	new file:   documentation/docs/getting-started/intro.mdx
	new file:   documentation/docs/getting-started/quickstart/docker-compose-config/opal-client.mdx
	new file:   documentation/docs/getting-started/quickstart/docker-compose-config/opal-server.mdx
	new file:   documentation/docs/getting-started/quickstart/docker-compose-config/overview.mdx
	new file:   documentation/docs/getting-started/quickstart/docker-compose-config/postgres-database.mdx
	new file:   documentation/docs/getting-started/quickstart/opal-playground/overview.mdx
	new file:   documentation/docs/getting-started/quickstart/opal-playground/publishing-data-update.mdx
	new file:   documentation/docs/getting-started/quickstart/opal-playground/run-server-and-client.mdx
	new file:   documentation/docs/getting-started/quickstart/opal-playground/send-queries-to-opa.mdx
	new file:   documentation/docs/getting-started/quickstart/opal-playground/updating-the-policy.mdx
	new file:   documentation/docs/getting-started/running-opal/as-python-package/opal-client-setup.mdx
	new file:   documentation/docs/getting-started/running-opal/as-python-package/opal-server-setup.mdx
	new file:   documentation/docs/getting-started/running-opal/as-python-package/overview.mdx
	new file:   documentation/docs/getting-started/running-opal/as-python-package/running-in-prod.mdx
	new file:   documentation/docs/getting-started/running-opal/as-python-package/secure-mode-setup.mdx
	new file:   documentation/docs/getting-started/running-opal/config-variables.mdx
	new file:   documentation/docs/getting-started/running-opal/download-docker-images.mdx
	new file:   documentation/docs/getting-started/running-opal/overview.mdx
	new file:   documentation/docs/getting-started/running-opal/run-docker-containers.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/data-topics.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/get-client-image.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/lets-run-the-client.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/obtain-jwt-token.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/opa-runner-parameters.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/server-uri.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-client/standalone-opa-uri.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/broadcast-interface.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/data-sources.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/get-server-image.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/policy-repo-location.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/policy-repo-syncing.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/putting-all-together.mdx
	new file:   documentation/docs/getting-started/running-opal/run-opal-server/security-parameters.mdx
	new file:   documentation/docs/getting-started/running-opal/troubleshooting.mdx
	new file:   documentation/docs/getting-started/tldr.mdx
	new file:   documentation/docs/opal-plus/deploy.mdx
	new file:   documentation/docs/opal-plus/features.mdx
	new file:   documentation/docs/opal-plus/introduction.mdx
	new file:   documentation/docs/opal-plus/troubleshooting.mdx
	new file:   documentation/docs/overview/_security.mdx
	new file:   documentation/docs/overview/architecture.mdx
	new file:   documentation/docs/overview/design.mdx
	new file:   documentation/docs/overview/modules.mdx
	new file:   documentation/docs/overview/scopes.md
	new file:   documentation/docs/release-updates.mdx
	new file:   documentation/docs/tutorials/_configure_backbone_pubsub.mdx
	new file:   documentation/docs/tutorials/cedar.mdx
	new file:   documentation/docs/tutorials/configure_external_data_sources.mdx
	new file:   documentation/docs/tutorials/configure_opal.mdx
	new file:   documentation/docs/tutorials/healthcheck_policy_and_update_callbacks.mdx
	new file:   documentation/docs/tutorials/helm-chart-for-kubernetes.mdx
	new file:   documentation/docs/tutorials/install_as_python_packages.mdx
	new file:   documentation/docs/tutorials/monitoring_opal.mdx
	new file:   documentation/docs/tutorials/run_opal_with_kafka.mdx
	new file:   documentation/docs/tutorials/run_opal_with_pulsar.mdx
	new file:   documentation/docs/tutorials/track_a_git_repo.mdx
	new file:   documentation/docs/tutorials/track_an_api_bundle_server.mdx
	new file:   documentation/docs/tutorials/trigger_data_updates.mdx
	new file:   documentation/docs/tutorials/use_self_signed_certificates.mdx
	new file:   documentation/docs/tutorials/write_your_own_fetch_provider.mdx
	new file:   documentation/docs/welcome.mdx
	new file:   documentation/docusaurus.config.js
	new file:   documentation/package-lock.json
	new file:   documentation/package.json
	new file:   documentation/sidebars.js
	new file:   documentation/src/css/custom.scss
	new file:   documentation/src/css/prism-theme.js
	new file:   documentation/static/.nojekyll
	new file:   documentation/static/img/FAQ-1.png
	new file:   documentation/static/img/favicon.ico
	new file:   documentation/static/img/opal.png
	new file:   documentation/static/img/opal_plus.png
	new file:   packages/__packaging__.py
	new file:   packages/opal-client/opal_client/__init__.py
	new file:   packages/opal-client/opal_client/callbacks/__init__.py
	new file:   packages/opal-client/opal_client/callbacks/api.py
	new file:   packages/opal-client/opal_client/callbacks/register.py
	new file:   packages/opal-client/opal_client/callbacks/reporter.py
	new file:   packages/opal-client/opal_client/cli.py
	new file:   packages/opal-client/opal_client/client.py
	new file:   packages/opal-client/opal_client/config.py
	new file:   packages/opal-client/opal_client/data/__init__.py
	new file:   packages/opal-client/opal_client/data/api.py
	new file:   packages/opal-client/opal_client/data/fetcher.py
	new file:   packages/opal-client/opal_client/data/rpc.py
	new file:   packages/opal-client/opal_client/data/updater.py
	new file:   packages/opal-client/opal_client/engine/__init__.py
	new file:   packages/opal-client/opal_client/engine/healthcheck/example-transaction.json
	new file:   packages/opal-client/opal_client/engine/healthcheck/opal.rego
	new file:   packages/opal-client/opal_client/engine/logger.py
	new file:   packages/opal-client/opal_client/engine/options.py
	new file:   packages/opal-client/opal_client/engine/runner.py
	new file:   packages/opal-client/opal_client/limiter.py
	new file:   packages/opal-client/opal_client/logger.py
	new file:   packages/opal-client/opal_client/main.py
	new file:   packages/opal-client/opal_client/policy/__init__.py
	new file:   packages/opal-client/opal_client/policy/api.py
	new file:   packages/opal-client/opal_client/policy/fetcher.py
	new file:   packages/opal-client/opal_client/policy/options.py
	new file:   packages/opal-client/opal_client/policy/topics.py
	new file:   packages/opal-client/opal_client/policy/updater.py
	new file:   packages/opal-client/opal_client/policy_store/__init__.py
	new file:   packages/opal-client/opal_client/policy_store/api.py
	new file:   packages/opal-client/opal_client/policy_store/base_policy_store_client.py
	new file:   packages/opal-client/opal_client/policy_store/cedar_client.py
	new file:   packages/opal-client/opal_client/policy_store/mock_policy_store_client.py
	new file:   packages/opal-client/opal_client/policy_store/opa_client.py
	new file:   packages/opal-client/opal_client/policy_store/policy_store_client_factory.py
	new file:   packages/opal-client/opal_client/policy_store/schemas.py
	new file:   packages/opal-client/opal_client/tests/__init__.py
	new file:   packages/opal-client/opal_client/tests/data_updater_test.py
	new file:   packages/opal-client/opal_client/tests/opa_client_test.py
	new file:   packages/opal-client/opal_client/tests/server_to_client_intergation_test.py
	new file:   packages/opal-client/opal_client/utils.py
	new file:   packages/opal-client/requires.txt
	new file:   packages/opal-client/setup.py
	new file:   packages/opal-common/opal_common/__init__.py
	new file:   packages/opal-common/opal_common/async_utils.py
	new file:   packages/opal-common/opal_common/authentication/__init__.py
	new file:   packages/opal-common/opal_common/authentication/authz.py
	new file:   packages/opal-common/opal_common/authentication/casting.py
	new file:   packages/opal-common/opal_common/authentication/deps.py
	new file:   packages/opal-common/opal_common/authentication/signer.py
	new file:   packages/opal-common/opal_common/authentication/tests/__init__.py
	new file:   packages/opal-common/opal_common/authentication/tests/jwt_signer_test.py
	new file:   packages/opal-common/opal_common/authentication/types.py
	new file:   packages/opal-common/opal_common/authentication/verifier.py
	new file:   packages/opal-common/opal_common/cli/__init__.py
	new file:   packages/opal-common/opal_common/cli/commands.py
	new file:   packages/opal-common/opal_common/cli/docs.py
	new file:   packages/opal-common/opal_common/cli/typer_app.py
	new file:   packages/opal-common/opal_common/confi/README.md
	new file:   packages/opal-common/opal_common/confi/__init__.py
	new file:   packages/opal-common/opal_common/confi/cli.py
	new file:   packages/opal-common/opal_common/confi/confi.py
	new file:   packages/opal-common/opal_common/confi/types.py
	new file:   packages/opal-common/opal_common/config.py
	new file:   packages/opal-common/opal_common/corn_utils.py
	new file:   packages/opal-common/opal_common/emport.py
	new file:   packages/opal-common/opal_common/engine/__init__.py
	new file:   packages/opal-common/opal_common/engine/parsing.py
	new file:   packages/opal-common/opal_common/engine/paths.py
	new file:   packages/opal-common/opal_common/engine/py.typed
	new file:   packages/opal-common/opal_common/engine/tests/fixtures/invalid-package.rego
	new file:   packages/opal-common/opal_common/engine/tests/fixtures/jwt.rego
	new file:   packages/opal-common/opal_common/engine/tests/fixtures/no-package.rego
	new file:   packages/opal-common/opal_common/engine/tests/fixtures/play.rego
	new file:   packages/opal-common/opal_common/engine/tests/fixtures/rbac.rego
	new file:   packages/opal-common/opal_common/engine/tests/parsing_test.py
	new file:   packages/opal-common/opal_common/engine/tests/paths_test.py
	new file:   packages/opal-common/opal_common/fetcher/__init__.py
	new file:   packages/opal-common/opal_common/fetcher/engine/__init__.py
	new file:   packages/opal-common/opal_common/fetcher/engine/base_fetching_engine.py
	new file:   packages/opal-common/opal_common/fetcher/engine/core_callbacks.py
	new file:   packages/opal-common/opal_common/fetcher/engine/fetch_worker.py
	new file:   packages/opal-common/opal_common/fetcher/engine/fetching_engine.py
	new file:   packages/opal-common/opal_common/fetcher/events.py
	new file:   packages/opal-common/opal_common/fetcher/fetch_provider.py
	new file:   packages/opal-common/opal_common/fetcher/fetcher_register.py
	new file:   packages/opal-common/opal_common/fetcher/logger.py
	new file:   packages/opal-common/opal_common/fetcher/providers/__init__.py
	new file:   packages/opal-common/opal_common/fetcher/providers/fastapi_rpc_fetch_provider.py
	new file:   packages/opal-common/opal_common/fetcher/providers/http_fetch_provider.py
	new file:   packages/opal-common/opal_common/fetcher/tests/__init__.py
	new file:   packages/opal-common/opal_common/fetcher/tests/failure_handler_test.py
	new file:   packages/opal-common/opal_common/fetcher/tests/http_fetch_test.py
	new file:   packages/opal-common/opal_common/fetcher/tests/rpc_fetch_test.py
	new file:   packages/opal-common/opal_common/git_utils/__init__.py
	new file:   packages/opal-common/opal_common/git_utils/branch_tracker.py
	new file:   packages/opal-common/opal_common/git_utils/bundle_maker.py
	new file:   packages/opal-common/opal_common/git_utils/bundle_utils.py
	new file:   packages/opal-common/opal_common/git_utils/commit_viewer.py
	new file:   packages/opal-common/opal_common/git_utils/diff_viewer.py
	new file:   packages/opal-common/opal_common/git_utils/env.py
	new file:   packages/opal-common/opal_common/git_utils/exceptions.py
	new file:   packages/opal-common/opal_common/git_utils/repo_cloner.py
	new file:   packages/opal-common/opal_common/git_utils/tar_file_to_local_git_extractor.py
	new file:   packages/opal-common/opal_common/git_utils/tests/branch_tracker_test.py
	new file:   packages/opal-common/opal_common/git_utils/tests/bundle_maker_test.py
	new file:   packages/opal-common/opal_common/git_utils/tests/commit_viewer_test.py
	new file:   packages/opal-common/opal_common/git_utils/tests/conftest.py
	new file:   packages/opal-common/opal_common/git_utils/tests/diff_viewer_test.py
	new file:   packages/opal-common/opal_common/git_utils/tests/repo_cloner_test.py
	new file:   packages/opal-common/opal_common/git_utils/tests/repo_watcher_test.py
	new file:   packages/opal-common/opal_common/http_utils.py
	new file:   packages/opal-common/opal_common/logger.py
	new file:   packages/opal-common/opal_common/logging_utils/__init__.py
	new file:   packages/opal-common/opal_common/logging_utils/decorators.py
	new file:   packages/opal-common/opal_common/logging_utils/filter.py
	new file:   packages/opal-common/opal_common/logging_utils/formatter.py
	new file:   packages/opal-common/opal_common/logging_utils/intercept.py
	new file:   packages/opal-common/opal_common/logging_utils/thirdparty.py
	new file:   packages/opal-common/opal_common/middleware.py
	new file:   packages/opal-common/opal_common/monitoring/__init__.py
	new file:   packages/opal-common/opal_common/monitoring/apm.py
	new file:   packages/opal-common/opal_common/monitoring/metrics.py
	new file:   packages/opal-common/opal_common/paths.py
	new file:   packages/opal-common/opal_common/schemas/__init__.py
	new file:   packages/opal-common/opal_common/schemas/data.py
	new file:   packages/opal-common/opal_common/schemas/policy.py
	new file:   packages/opal-common/opal_common/schemas/policy_source.py
	new file:   packages/opal-common/opal_common/schemas/scopes.py
	new file:   packages/opal-common/opal_common/schemas/security.py
	new file:   packages/opal-common/opal_common/schemas/store.py
	new file:   packages/opal-common/opal_common/schemas/webhook.py
	new file:   packages/opal-common/opal_common/security/__init__.py
	new file:   packages/opal-common/opal_common/security/sslcontext.py
	new file:   packages/opal-common/opal_common/security/tarsafe.py
	new file:   packages/opal-common/opal_common/sources/__init__.py
	new file:   packages/opal-common/opal_common/sources/api_policy_source.py
	new file:   packages/opal-common/opal_common/sources/base_policy_source.py
	new file:   packages/opal-common/opal_common/sources/git_policy_source.py
	new file:   packages/opal-common/opal_common/synchronization/__init__.py
	new file:   packages/opal-common/opal_common/synchronization/expiring_redis_lock.py
	new file:   packages/opal-common/opal_common/synchronization/named_lock.py
	new file:   packages/opal-common/opal_common/tests/__init__.py
	new file:   packages/opal-common/opal_common/tests/path_utils_test.py
	new file:   packages/opal-common/opal_common/tests/test_utils.py
	new file:   packages/opal-common/opal_common/tests/url_utils_test.py
	new file:   packages/opal-common/opal_common/topics/__init__.py
	new file:   packages/opal-common/opal_common/topics/listener.py
	new file:   packages/opal-common/opal_common/topics/publisher.py
	new file:   packages/opal-common/opal_common/topics/utils.py
	new file:   packages/opal-common/opal_common/urls.py
	new file:   packages/opal-common/opal_common/utils.py
	new file:   packages/opal-common/requires.txt
	new file:   packages/opal-common/setup.py
	new file:   packages/opal-server/opal_server/__init__.py
	new file:   packages/opal-server/opal_server/cli.py
	new file:   packages/opal-server/opal_server/config.py
	new file:   packages/opal-server/opal_server/data/__init__.py
	new file:   packages/opal-server/opal_server/data/api.py
	new file:   packages/opal-server/opal_server/data/data_update_publisher.py
	new file:   packages/opal-server/opal_server/data/tests/test_data_update_publisher.py
	new file:   packages/opal-server/opal_server/git_fetcher.py
	new file:   packages/opal-server/opal_server/loadlimiting.py
	new file:   packages/opal-server/opal_server/main.py
	new file:   packages/opal-server/opal_server/policy/__init__.py
	new file:   packages/opal-server/opal_server/policy/bundles/__init__.py
	new file:   packages/opal-server/opal_server/policy/bundles/api.py
	new file:   packages/opal-server/opal_server/policy/watcher/__init__.py
	new file:   packages/opal-server/opal_server/policy/watcher/callbacks.py
	new file:   packages/opal-server/opal_server/policy/watcher/factory.py
	new file:   packages/opal-server/opal_server/policy/watcher/task.py
	new file:   packages/opal-server/opal_server/policy/webhook/__init__.py
	new file:   packages/opal-server/opal_server/policy/webhook/api.py
	new file:   packages/opal-server/opal_server/policy/webhook/deps.py
	new file:   packages/opal-server/opal_server/policy/webhook/listener.py
	new file:   packages/opal-server/opal_server/publisher.py
	new file:   packages/opal-server/opal_server/pubsub.py
	new file:   packages/opal-server/opal_server/redis_utils.py
	new file:   packages/opal-server/opal_server/scopes/__init__.py
	new file:   packages/opal-server/opal_server/scopes/api.py
	new file:   packages/opal-server/opal_server/scopes/loader.py
	new file:   packages/opal-server/opal_server/scopes/scope_repository.py
	new file:   packages/opal-server/opal_server/scopes/service.py
	new file:   packages/opal-server/opal_server/scopes/task.py
	new file:   packages/opal-server/opal_server/security/__init__.py
	new file:   packages/opal-server/opal_server/security/api.py
	new file:   packages/opal-server/opal_server/security/jwks.py
	new file:   packages/opal-server/opal_server/server.py
	new file:   packages/opal-server/opal_server/statistics.py
	new file:   packages/opal-server/opal_server/tests/policy_repo_webhook_test.py
	new file:   packages/opal-server/requires.txt
	new file:   packages/opal-server/setup.py
	new file:   packages/requires.txt
	new file:   pytest.ini
	new file:   requirements.txt
	new file:   scripts/gunicorn_conf.py
	new file:   scripts/start.sh
	new file:   scripts/wait-for.sh
This commit makes several changes to improve the OPAL server configuration:

- Switch from using a pre-built image to building from source
- Reduce server replicas and worker count for better debugging
- Update policy repo URL to use the example repo
- Disable OPAL client in the test configuration
- Add container names for easier identification
- Enable remote debugging with debugpy

These changes enhance the development and testing environment, allowing
for easier troubleshooting and a more consistent setup across different
machines. The addition of remote debugging capabilities will significantly
improve the ability to diagnose and fix issues in the OPAL server.
	modified:   ari/docker-py.py
	new file:   new_pytest_env/b.py
	renamed:    ari/docker-compose.yml -> new_pytest_env/docker-compose.yml
	new file:   new_pytest_env/docker-py copy.py
	new file:   new_pytest_env/gitea_docker_py.py
	new file:   new_pytest_env/github_clone_to_gitea.py
	renamed:    ari/docker-py.py -> new_pytest_env/opal_docker_py.py
	new file:   opal-example-policy-repo
ariWeinberg and others added 29 commits January 6, 2025 03:13
commit 4f3dc92
Author: Israel Weinberg <israel@phone.do>
Date:   Mon Jan 6 14:32:34 2025 +0200

    feat: add global exception handler to improve error reporting in tests
@izzywdev
izzywdev merged commit 30809a7 into e2e-deploy Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants